mirror of
https://github.com/Part-DB/Part-DB-server.git
synced 2026-03-03 05:49:36 +00:00
Enhance KiCad integration: API v2, batch EDA editing, field export control (#1241)
* Add stock quantity, datasheet URL, and HTTP caching to KiCad API - Add Stock field showing total available quantity across all part lots - Add Storage Location field when parts have stored locations - Resolve actual datasheet PDF from attachments (by type name, attachment name, or first PDF) instead of always linking to Part-DB page - Keep Part-DB page URL as separate "Part-DB URL" field - Add ETag and Cache-Control headers to all KiCad API endpoints - Support conditional requests (If-None-Match) returning 304 - Categories/part lists cached 5 min, part details cached 1 min * Add KiCadHelper unit tests and fix PDF detection for external URLs - Add comprehensive KiCadHelperTest with 14 test cases covering: - Stock quantity calculation (zero, single lot, multiple lots) - Stock exclusion of expired and unknown-quantity lots - Storage location display (present, absent, multiple) - Datasheet URL resolution by type name, attachment name, PDF extension - Datasheet fallback to Part-DB URL when no match - "Data sheet" (with space) name variant matching - Fix PDF extension detection for external attachments (getExtension() returns null for external-only attachments, now also parses URL path) * Fix 304 response body, parse_url safety, and location/stock consistency - Use empty Response instead of JsonResponse(null) for 304 Not Modified to avoid sending "null" as response body - Guard parse_url() result with is_string() since it can return false for malformed URLs - Move storage location tracking inside the availability check so expired and unknown-quantity lots don't contribute locations * Fix testPartDetailsPart2 to actually test Part 2 The test was requesting /parts/1.json instead of /parts/2.json and had Part 1's expected data. Now tests Part 2 which inherits EDA info from its category and footprint, verifying the inheritance behavior. * Use Symfony's built-in ETag handling for HTTP caching Replace manual If-None-Match comparison with Response::setEtag() and Response::isNotModified(), which properly handles ETag quoting, weak vs strong comparison, and 304 response cleanup. Fixes PHPStan return type error and CI test failures. * Add configurable KiCad field export for part parameters Add a kicad_export checkbox to parameters, allowing users to control which specifications appear as fields in the KiCad HTTP library API. Parameters with kicad_export enabled are included using their formatted value, without overwriting hardcoded fields like description or Stock. * Add partdb:kicad:populate command for bulk KiCad path assignment Console command that populates KiCad footprint/symbol paths on Footprint and Category entities based on name-to-library mappings. Supports dry-run, force overwrite, and list modes. Includes 130+ footprint mappings and 30+ category symbol mappings for KiCad 9.x standard libraries. * Add CSV import support for EDA/KiCad fields Add user-friendly column aliases (kicad_symbol, kicad_footprint, kicad_reference, kicad_value, eda_exclude_bom, etc.) to the CSV import system. Users can now bulk-set KiCad symbols, footprints, and other EDA metadata via CSV/Excel import without knowing the internal dot notation. * Add batch EDA field editing from parts table Users can now select multiple parts in any parts table and batch-edit their EDA/KiCad fields (symbol, footprint, reference prefix, value, visibility, exclude from BOM/board/sim). Each field has an "Apply" checkbox so users control exactly which fields are changed. * Remove unused counter variable in BatchEdaController * Fix PHPStan errors in PopulateKicadCommand and BatchEdaController Add @var type annotations for Doctrine repository findAll() calls so PHPStan can resolve getEdaInfo() on Footprint/Category entities. Fix array return type for numeric-string keys and add explicit callback to array_filter to satisfy strict rules. * Fix batch EDA edit: required validation and pre-populate shared values - Add required=false to TriStateCheckboxType fields so HTML5 validation doesn't force users to check visibility/BOM/board checkboxes - Pre-populate form fields when all selected parts share the same EDA value, so users can see current state before editing * Add KiCad API v2, orderdetail export control, EDA status indicator, BOM improvements - Add KiCad API v2 endpoints (/kicad-api/v2) with volatile field support for stock and storage location (shown but not saved to schematic) - Add kicad_export flag to Orderdetail entity for per-supplier SPN control (backward compatible: if no flag set, all SPNs exported as before) - Add EDA completeness indicator column in parts datatable (bolt icon) - Add ?minimal=true query param for faster category parts loading - Improve category descriptions (use comment instead of URL when available) - Improve BOM importer multi-footprint support: merge entries by Part-DB part ID when linked, tracking footprint variants in comments - Fix KiCost manf/manf# fields always present (not conditional on orderdetails) - Fix duplicate getEdaInfo() call in shouldPartBeVisible - Consolidate supplier SPN and KiCost field generation into single loop * Fix kicad_export column default for SQLite compatibility Add options default to ORM column definition so schema:update works correctly on SQLite test databases. * Make EDA status bolt icon clickable to open EDA settings tab * Fix EDA bolt link to correctly open EDA tab via data-turbo=false * Add configurable datasheet URL mode for KiCad API New setting "Datasheet field links to PDF" in KiCad EDA settings. When enabled (default), the datasheet field resolves to the actual PDF attachment URL. When disabled, it links to the Part-DB page (old behavior). Configurable via settings UI or EDA_KICAD_DATASHEET_AS_PDF env var. * Fix settings crash when upgrading: make datasheetAsPdf nullable The settings bundle stores values in the database. When upgrading from a version without datasheetAsPdf, the stored JSON lacks this key, causing a TypeError when assigning null to a non-nullable bool. Making it nullable with a fallback in KiCadHelper fixes the upgrade path. * Add functional tests for KiCad API v2 and batch EDA controller - KiCadApiV2ControllerTest: root, categories, parts, volatile fields, v1 vs v2 comparison, cache headers, 304 conditional request, auth - BatchEdaControllerTest: page load, empty redirect, form submission * Fix test failures: correct ids format and anonymous access assertion * Improve test coverage for BatchEdaController Add tests for: applying all EDA fields at once, custom redirect URL, and verifying unchecked fields are skipped. * Address PR review: rename to eda_visibility, merge migrations, API versioning Changes based on jbtronics' review of PR #1241: - Rename kicad_export -> eda_visibility (entities, forms, templates, translations, tests) with nullable bool for system default support - Merge two database migrations into one (Version20260211000000) - Rename createCachedJsonResponse -> createCacheableJsonResponse - Change bool $apiV2 -> int $apiVersion with version validation - EDA visibility field only shown for part parameters, not other entities - PopulateKicadCommand: check alternative names of footprints/categories - PopulateKicadCommand: support external JSON mapping file (--mapping-file) - Ship default mappings JSON at contrib/kicad-populate/default_mappings.json - Add system-wide defaultEdaVisibility setting in KiCadEDASettings - Add KiCad HTTP Library v2 spec link in controller docs * Fix duplicate loadMappingFile method causing PHP fatal error * Add tests for mapping file and alternative name matching, update populate command docs Add 5 new tests for PopulateKicadCommand covering: - Custom mapping file overriding defaults - Invalid JSON mapping file error handling - Missing mapping file error handling - Footprint alternative name matching - Category alternative name matching Update contrib README to document --mapping-file option, alternative name matching, and custom JSON mapping format. * Split out KiCad API v2 into separate PR as requested by maintainer Remove v2 controller, tests, and volatile field support from this PR. The v2 API will be submitted as a separate PR for focused discussion. * Improve test coverage for KiCadHelper and PopulateKicadCommand KiCadHelper: Add tests for orderdetail eda_visibility filtering, backward compatibility when no flags set, manufacturer/KiCost fields, and parameter with empty name skipping. PopulateKicadCommand: Add tests for mapping file with both footprints and categories sections, and mapping file with only categories. * Load populate Kicad default mappings from json file * Moved kicad:populate documentation to central docs * Added introduced column to PartTableColumns to make it configurable in the settings * Use TristateCheckboxes for parameter and orderdetail types * Fixed translation keys * Split up default eda visibility for parameters and purchase infos --------- Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
This commit is contained in:
Sebastian Almberg
GitHub
parent
46617f01a4
commit
b9d940ae33
30 changed files with 2920 additions and 67 deletions
|
|
@ -23,6 +23,7 @@ declare(strict_types=1);
|
|||
|
||||
namespace App\Services\EDA;
|
||||
|
||||
use App\Entity\Attachments\Attachment;
|
||||
use App\Entity\Parts\Category;
|
||||
use App\Entity\Parts\Footprint;
|
||||
use App\Entity\Parts\Part;
|
||||
|
|
@ -43,6 +44,9 @@ class KiCadHelper
|
|||
/** @var int The maximum level of the shown categories. 0 Means only the top level categories are shown. -1 means only a single one containing */
|
||||
private readonly int $category_depth;
|
||||
|
||||
/** @var bool Whether to resolve actual datasheet PDF URLs (true) or use Part-DB page links (false) */
|
||||
private readonly bool $datasheetAsPdf;
|
||||
|
||||
public function __construct(
|
||||
private readonly NodesListBuilder $nodesListBuilder,
|
||||
private readonly TagAwareCacheInterface $kicadCache,
|
||||
|
|
@ -51,9 +55,10 @@ class KiCadHelper
|
|||
private readonly UrlGeneratorInterface $urlGenerator,
|
||||
private readonly EntityURLGenerator $entityURLGenerator,
|
||||
private readonly TranslatorInterface $translator,
|
||||
KiCadEDASettings $kiCadEDASettings,
|
||||
private readonly KiCadEDASettings $kiCadEDASettings,
|
||||
) {
|
||||
$this->category_depth = $kiCadEDASettings->categoryDepth;
|
||||
$this->datasheetAsPdf = $kiCadEDASettings->datasheetAsPdf ?? true;
|
||||
}
|
||||
|
||||
/**
|
||||
|
|
@ -115,11 +120,16 @@ class KiCadHelper
|
|||
}
|
||||
|
||||
//Format the category for KiCAD
|
||||
// Use the category comment as description if available, otherwise use the Part-DB URL
|
||||
$description = $category->getComment();
|
||||
if ($description === null || $description === '') {
|
||||
$description = $this->entityURLGenerator->listPartsURL($category);
|
||||
}
|
||||
|
||||
$result[] = [
|
||||
'id' => (string)$category->getId(),
|
||||
'name' => $category->getFullPath('/'),
|
||||
//Show the category link as the category description, this also fixes an segfault in KiCad see issue #878
|
||||
'description' => $this->entityURLGenerator->listPartsURL($category),
|
||||
'description' => $description,
|
||||
];
|
||||
}
|
||||
|
||||
|
|
@ -131,11 +141,13 @@ class KiCadHelper
|
|||
* Returns an array of objects containing all parts for the given category in the format required by KiCAD.
|
||||
* The result is cached for performance and invalidated on category or part changes.
|
||||
* @param Category|null $category
|
||||
* @param bool $minimal If true, only return id and name (faster for symbol chooser listing)
|
||||
* @return array
|
||||
*/
|
||||
public function getCategoryParts(?Category $category): array
|
||||
public function getCategoryParts(?Category $category, bool $minimal = false): array
|
||||
{
|
||||
return $this->kicadCache->get('kicad_category_parts_'.($category?->getID() ?? 0) . '_' . $this->category_depth,
|
||||
$cacheKey = 'kicad_category_parts_'.($category?->getID() ?? 0) . '_' . $this->category_depth . ($minimal ? '_min' : '');
|
||||
return $this->kicadCache->get($cacheKey,
|
||||
function (ItemInterface $item) use ($category) {
|
||||
$item->tag([
|
||||
$this->tagGenerator->getElementTypeCacheTag(Category::class),
|
||||
|
|
@ -198,14 +210,22 @@ class KiCadHelper
|
|||
$result["fields"]["value"] = $this->createField($part->getEdaInfo()->getValue() ?? $part->getName(), true);
|
||||
$result["fields"]["keywords"] = $this->createField($part->getTags());
|
||||
|
||||
//Use the part info page as datasheet link. It must be an absolute URL.
|
||||
$result["fields"]["datasheet"] = $this->createField(
|
||||
$this->urlGenerator->generate(
|
||||
'part_info',
|
||||
['id' => $part->getId()],
|
||||
UrlGeneratorInterface::ABSOLUTE_URL)
|
||||
//Use the part info page as Part-DB link. It must be an absolute URL.
|
||||
$partUrl = $this->urlGenerator->generate(
|
||||
'part_info',
|
||||
['id' => $part->getId()],
|
||||
UrlGeneratorInterface::ABSOLUTE_URL
|
||||
);
|
||||
|
||||
//Try to find an actual datasheet attachment (configurable: PDF URL vs Part-DB page link)
|
||||
if ($this->datasheetAsPdf) {
|
||||
$datasheetUrl = $this->findDatasheetUrl($part);
|
||||
$result["fields"]["datasheet"] = $this->createField($datasheetUrl ?? $partUrl);
|
||||
} else {
|
||||
$result["fields"]["datasheet"] = $this->createField($partUrl);
|
||||
}
|
||||
$result["fields"]["Part-DB URL"] = $this->createField($partUrl);
|
||||
|
||||
//Add basic fields
|
||||
$result["fields"]["description"] = $this->createField($part->getDescription());
|
||||
if ($part->getCategory() !== null) {
|
||||
|
|
@ -245,32 +265,7 @@ class KiCadHelper
|
|||
$result["fields"]["Part-DB IPN"] = $this->createField($part->getIpn());
|
||||
}
|
||||
|
||||
// Add supplier information from orderdetails (include obsolete orderdetails)
|
||||
if ($part->getOrderdetails(false)->count() > 0) {
|
||||
$supplierCounts = [];
|
||||
|
||||
foreach ($part->getOrderdetails(false) as $orderdetail) {
|
||||
if ($orderdetail->getSupplier() !== null && $orderdetail->getSupplierPartNr() !== '') {
|
||||
$supplierName = $orderdetail->getSupplier()->getName();
|
||||
|
||||
$supplierName .= " SPN"; // Append "SPN" to the supplier name to indicate Supplier Part Number
|
||||
|
||||
if (!isset($supplierCounts[$supplierName])) {
|
||||
$supplierCounts[$supplierName] = 0;
|
||||
}
|
||||
$supplierCounts[$supplierName]++;
|
||||
|
||||
// Create field name with sequential number if more than one from same supplier (e.g. "Mouser", "Mouser 2", etc.)
|
||||
$fieldName = $supplierCounts[$supplierName] > 1
|
||||
? $supplierName . ' ' . $supplierCounts[$supplierName]
|
||||
: $supplierName;
|
||||
|
||||
$result["fields"][$fieldName] = $this->createField($orderdetail->getSupplierPartNr());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//Add fields for KiCost:
|
||||
//Add KiCost manufacturer fields (always present, independent of orderdetails)
|
||||
if ($part->getManufacturer() !== null) {
|
||||
$result["fields"]["manf"] = $this->createField($part->getManufacturer()->getName());
|
||||
}
|
||||
|
|
@ -278,13 +273,74 @@ class KiCadHelper
|
|||
$result['fields']['manf#'] = $this->createField($part->getManufacturerProductNumber());
|
||||
}
|
||||
|
||||
//For each supplier, add a field with the supplier name and the supplier part number for KiCost
|
||||
if ($part->getOrderdetails(false)->count() > 0) {
|
||||
foreach ($part->getOrderdetails(false) as $orderdetail) {
|
||||
// Add supplier information from orderdetails (include obsolete orderdetails)
|
||||
// If any orderdetail has eda_visibility explicitly set to true, only export those;
|
||||
// otherwise export all (backward compat when no flags are set)
|
||||
$allOrderdetails = $part->getOrderdetails(false);
|
||||
if ($allOrderdetails->count() > 0) {
|
||||
$hasExplicitEdaVisibility = false;
|
||||
foreach ($allOrderdetails as $od) {
|
||||
if ($od->isEdaVisibility() !== null) {
|
||||
$hasExplicitEdaVisibility = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
$supplierCounts = [];
|
||||
foreach ($allOrderdetails as $orderdetail) {
|
||||
if ($orderdetail->getSupplier() !== null && $orderdetail->getSupplierPartNr() !== '') {
|
||||
$fieldName = mb_strtolower($orderdetail->getSupplier()->getName()) . '#';
|
||||
// When explicit flags exist, filter by resolved visibility
|
||||
$resolvedVisibility = $orderdetail->isEdaVisibility() ?? $this->kiCadEDASettings->defaultOrderdetailsVisibility;
|
||||
if ($hasExplicitEdaVisibility && !$resolvedVisibility) {
|
||||
continue;
|
||||
}
|
||||
|
||||
$supplierName = $orderdetail->getSupplier()->getName() . ' SPN';
|
||||
|
||||
if (!isset($supplierCounts[$supplierName])) {
|
||||
$supplierCounts[$supplierName] = 0;
|
||||
}
|
||||
$supplierCounts[$supplierName]++;
|
||||
|
||||
// Create field name with sequential number if more than one from same supplier
|
||||
$fieldName = $supplierCounts[$supplierName] > 1
|
||||
? $supplierName . ' ' . $supplierCounts[$supplierName]
|
||||
: $supplierName;
|
||||
|
||||
$result["fields"][$fieldName] = $this->createField($orderdetail->getSupplierPartNr());
|
||||
|
||||
//Also add a KiCost-compatible field (supplier_name# = SPN)
|
||||
$kicostFieldName = mb_strtolower($orderdetail->getSupplier()->getName()) . '#';
|
||||
$result["fields"][$kicostFieldName] = $this->createField($orderdetail->getSupplierPartNr());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//Add stock quantity and storage locations (only count non-expired lots with known quantity)
|
||||
$totalStock = 0;
|
||||
$locations = [];
|
||||
foreach ($part->getPartLots() as $lot) {
|
||||
$isAvailable = !$lot->isInstockUnknown() && $lot->isExpired() !== true;
|
||||
if ($isAvailable) {
|
||||
$totalStock += $lot->getAmount();
|
||||
if ($lot->getAmount() > 0 && $lot->getStorageLocation() !== null) {
|
||||
$locations[] = $lot->getStorageLocation()->getName();
|
||||
}
|
||||
}
|
||||
}
|
||||
$result['fields']['Stock'] = $this->createField($totalStock);
|
||||
if ($locations !== []) {
|
||||
$result['fields']['Storage Location'] = $this->createField(implode(', ', array_unique($locations)));
|
||||
}
|
||||
|
||||
//Add parameters marked for EDA export (explicit true, or system default when null)
|
||||
foreach ($part->getParameters() as $parameter) {
|
||||
$paramVisibility = $parameter->isEdaVisibility() ?? $this->kiCadEDASettings->defaultParameterVisibility;
|
||||
if ($paramVisibility && $parameter->getName() !== '') {
|
||||
$fieldName = $parameter->getName();
|
||||
//Don't overwrite hardcoded fields
|
||||
if (!isset($result['fields'][$fieldName])) {
|
||||
$result['fields'][$fieldName] = $this->createField($parameter->getFormattedValue());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
|
@ -344,7 +400,7 @@ class KiCadHelper
|
|||
|
||||
//If the user set a visibility, then use it
|
||||
if ($eda_info->getVisibility() !== null) {
|
||||
return $part->getEdaInfo()->getVisibility();
|
||||
return $eda_info->getVisibility();
|
||||
}
|
||||
|
||||
//If the part has a category, then use the category visibility if possible
|
||||
|
|
@ -395,4 +451,64 @@ class KiCadHelper
|
|||
'visible' => $this->boolToKicadBool($visible),
|
||||
];
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Finds the URL to the actual datasheet file for the given part.
|
||||
* Searches attachments by type name, attachment name, and file extension.
|
||||
* @return string|null The datasheet URL, or null if no datasheet was found.
|
||||
*/
|
||||
private function findDatasheetUrl(Part $part): ?string
|
||||
{
|
||||
$firstPdf = null;
|
||||
|
||||
foreach ($part->getAttachments() as $attachment) {
|
||||
//Check if the attachment type name contains "datasheet"
|
||||
$typeName = $attachment->getAttachmentType()?->getName() ?? '';
|
||||
if (str_contains(mb_strtolower($typeName), 'datasheet')) {
|
||||
return $this->getAttachmentUrl($attachment);
|
||||
}
|
||||
|
||||
//Check if the attachment name contains "datasheet"
|
||||
$name = mb_strtolower($attachment->getName());
|
||||
if (str_contains($name, 'datasheet') || str_contains($name, 'data sheet')) {
|
||||
return $this->getAttachmentUrl($attachment);
|
||||
}
|
||||
|
||||
//Track first PDF as fallback (check internal extension or external URL path)
|
||||
if ($firstPdf === null) {
|
||||
$extension = $attachment->getExtension();
|
||||
if ($extension === null && $attachment->hasExternal()) {
|
||||
$urlPath = parse_url($attachment->getExternalPath(), PHP_URL_PATH);
|
||||
$extension = is_string($urlPath) ? strtolower(pathinfo($urlPath, PATHINFO_EXTENSION)) : null;
|
||||
}
|
||||
if ($extension === 'pdf') {
|
||||
$firstPdf = $attachment;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
//Use first PDF attachment as fallback
|
||||
if ($firstPdf !== null) {
|
||||
return $this->getAttachmentUrl($firstPdf);
|
||||
}
|
||||
|
||||
return null;
|
||||
}
|
||||
|
||||
/**
|
||||
* Returns an absolute URL for viewing the given attachment.
|
||||
* Prefers the external URL (direct link) over the internal view route.
|
||||
*/
|
||||
private function getAttachmentUrl(Attachment $attachment): string
|
||||
{
|
||||
if ($attachment->hasExternal()) {
|
||||
return $attachment->getExternalPath();
|
||||
}
|
||||
|
||||
return $this->urlGenerator->generate(
|
||||
'attachment_view',
|
||||
['id' => $attachment->getId()],
|
||||
UrlGeneratorInterface::ABSOLUTE_URL
|
||||
);
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue