Commit graph

85 commits

Author SHA1 Message Date
Jan Böhmer
74e5102943 Automatically detect the delimiter of generic BOM imports
The detectFields does this anyway, so use that guessed value further on
2026-03-15 21:35:38 +01:00
Jan Böhmer
7f8f5990a7 Fixed phpstan issues 2026-03-07 22:30:39 +01:00
Jan Böhmer
12a760d27e Correctly denormalize parent-child relationships in import, when only children not parent fields are given
This fixes issue #1272
2026-03-07 21:08:32 +01:00
Sebastian Almberg
b9d940ae33
Enhance KiCad integration: API v2, batch EDA editing, field export control (#1241)
* Add stock quantity, datasheet URL, and HTTP caching to KiCad API

- Add Stock field showing total available quantity across all part lots
- Add Storage Location field when parts have stored locations
- Resolve actual datasheet PDF from attachments (by type name, attachment
  name, or first PDF) instead of always linking to Part-DB page
- Keep Part-DB page URL as separate "Part-DB URL" field
- Add ETag and Cache-Control headers to all KiCad API endpoints
- Support conditional requests (If-None-Match) returning 304
- Categories/part lists cached 5 min, part details cached 1 min

* Add KiCadHelper unit tests and fix PDF detection for external URLs

- Add comprehensive KiCadHelperTest with 14 test cases covering:
  - Stock quantity calculation (zero, single lot, multiple lots)
  - Stock exclusion of expired and unknown-quantity lots
  - Storage location display (present, absent, multiple)
  - Datasheet URL resolution by type name, attachment name, PDF extension
  - Datasheet fallback to Part-DB URL when no match
  - "Data sheet" (with space) name variant matching
- Fix PDF extension detection for external attachments (getExtension()
  returns null for external-only attachments, now also parses URL path)

* Fix 304 response body, parse_url safety, and location/stock consistency

- Use empty Response instead of JsonResponse(null) for 304 Not Modified
  to avoid sending "null" as response body
- Guard parse_url() result with is_string() since it can return false
  for malformed URLs
- Move storage location tracking inside the availability check so
  expired and unknown-quantity lots don't contribute locations

* Fix testPartDetailsPart2 to actually test Part 2

The test was requesting /parts/1.json instead of /parts/2.json and had
Part 1's expected data. Now tests Part 2 which inherits EDA info from
its category and footprint, verifying the inheritance behavior.

* Use Symfony's built-in ETag handling for HTTP caching

Replace manual If-None-Match comparison with Response::setEtag() and
Response::isNotModified(), which properly handles ETag quoting, weak
vs strong comparison, and 304 response cleanup. Fixes PHPStan return
type error and CI test failures.

* Add configurable KiCad field export for part parameters

Add a kicad_export checkbox to parameters, allowing users to control
which specifications appear as fields in the KiCad HTTP library API.
Parameters with kicad_export enabled are included using their formatted
value, without overwriting hardcoded fields like description or Stock.

* Add partdb:kicad:populate command for bulk KiCad path assignment

Console command that populates KiCad footprint/symbol paths on Footprint
and Category entities based on name-to-library mappings. Supports dry-run,
force overwrite, and list modes. Includes 130+ footprint mappings and 30+
category symbol mappings for KiCad 9.x standard libraries.

* Add CSV import support for EDA/KiCad fields

Add user-friendly column aliases (kicad_symbol, kicad_footprint,
kicad_reference, kicad_value, eda_exclude_bom, etc.) to the CSV import
system. Users can now bulk-set KiCad symbols, footprints, and other EDA
metadata via CSV/Excel import without knowing the internal dot notation.

* Add batch EDA field editing from parts table

Users can now select multiple parts in any parts table and batch-edit
their EDA/KiCad fields (symbol, footprint, reference prefix, value,
visibility, exclude from BOM/board/sim). Each field has an "Apply"
checkbox so users control exactly which fields are changed.

* Remove unused counter variable in BatchEdaController

* Fix PHPStan errors in PopulateKicadCommand and BatchEdaController

Add @var type annotations for Doctrine repository findAll() calls so
PHPStan can resolve getEdaInfo() on Footprint/Category entities. Fix
array return type for numeric-string keys and add explicit callback to
array_filter to satisfy strict rules.

* Fix batch EDA edit: required validation and pre-populate shared values

- Add required=false to TriStateCheckboxType fields so HTML5 validation
  doesn't force users to check visibility/BOM/board checkboxes
- Pre-populate form fields when all selected parts share the same EDA
  value, so users can see current state before editing

* Add KiCad API v2, orderdetail export control, EDA status indicator, BOM improvements

- Add KiCad API v2 endpoints (/kicad-api/v2) with volatile field support
  for stock and storage location (shown but not saved to schematic)
- Add kicad_export flag to Orderdetail entity for per-supplier SPN control
  (backward compatible: if no flag set, all SPNs exported as before)
- Add EDA completeness indicator column in parts datatable (bolt icon)
- Add ?minimal=true query param for faster category parts loading
- Improve category descriptions (use comment instead of URL when available)
- Improve BOM importer multi-footprint support: merge entries by Part-DB
  part ID when linked, tracking footprint variants in comments
- Fix KiCost manf/manf# fields always present (not conditional on orderdetails)
- Fix duplicate getEdaInfo() call in shouldPartBeVisible
- Consolidate supplier SPN and KiCost field generation into single loop

* Fix kicad_export column default for SQLite compatibility

Add options default to ORM column definition so schema:update
works correctly on SQLite test databases.

* Make EDA status bolt icon clickable to open EDA settings tab

* Fix EDA bolt link to correctly open EDA tab via data-turbo=false

* Add configurable datasheet URL mode for KiCad API

New setting "Datasheet field links to PDF" in KiCad EDA settings.
When enabled (default), the datasheet field resolves to the actual
PDF attachment URL. When disabled, it links to the Part-DB page
(old behavior). Configurable via settings UI or EDA_KICAD_DATASHEET_AS_PDF env var.

* Fix settings crash when upgrading: make datasheetAsPdf nullable

The settings bundle stores values in the database. When upgrading from
a version without datasheetAsPdf, the stored JSON lacks this key,
causing a TypeError when assigning null to a non-nullable bool.
Making it nullable with a fallback in KiCadHelper fixes the upgrade path.

* Add functional tests for KiCad API v2 and batch EDA controller

- KiCadApiV2ControllerTest: root, categories, parts, volatile fields,
  v1 vs v2 comparison, cache headers, 304 conditional request, auth
- BatchEdaControllerTest: page load, empty redirect, form submission

* Fix test failures: correct ids format and anonymous access assertion

* Improve test coverage for BatchEdaController

Add tests for: applying all EDA fields at once, custom redirect URL,
and verifying unchecked fields are skipped.

* Address PR review: rename to eda_visibility, merge migrations, API versioning

Changes based on jbtronics' review of PR #1241:

- Rename kicad_export -> eda_visibility (entities, forms, templates,
  translations, tests) with nullable bool for system default support
- Merge two database migrations into one (Version20260211000000)
- Rename createCachedJsonResponse -> createCacheableJsonResponse
- Change bool $apiV2 -> int $apiVersion with version validation
- EDA visibility field only shown for part parameters, not other entities
- PopulateKicadCommand: check alternative names of footprints/categories
- PopulateKicadCommand: support external JSON mapping file (--mapping-file)
- Ship default mappings JSON at contrib/kicad-populate/default_mappings.json
- Add system-wide defaultEdaVisibility setting in KiCadEDASettings
- Add KiCad HTTP Library v2 spec link in controller docs

* Fix duplicate loadMappingFile method causing PHP fatal error

* Add tests for mapping file and alternative name matching, update populate command docs

Add 5 new tests for PopulateKicadCommand covering:
- Custom mapping file overriding defaults
- Invalid JSON mapping file error handling
- Missing mapping file error handling
- Footprint alternative name matching
- Category alternative name matching

Update contrib README to document --mapping-file option,
alternative name matching, and custom JSON mapping format.

* Split out KiCad API v2 into separate PR as requested by maintainer

Remove v2 controller, tests, and volatile field support from this PR.
The v2 API will be submitted as a separate PR for focused discussion.

* Improve test coverage for KiCadHelper and PopulateKicadCommand

KiCadHelper: Add tests for orderdetail eda_visibility filtering,
backward compatibility when no flags set, manufacturer/KiCost fields,
and parameter with empty name skipping.

PopulateKicadCommand: Add tests for mapping file with both footprints
and categories sections, and mapping file with only categories.

* Load populate Kicad default mappings from json file

* Moved kicad:populate documentation to central docs

* Added introduced column to PartTableColumns to make it configurable in the settings

* Use TristateCheckboxes for parameter and orderdetail types

* Fixed translation keys

* Split up default eda visibility for parameters and purchase infos

---------

Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
2026-03-01 22:10:13 +01:00
d-buchmann
97a74815d3
Fix fallback filename (#1238)
Some checks are pending
Build assets artifact / Build assets artifact (push) Waiting to run
Docker Image Build / docker (push) Waiting to run
Docker Image Build (FrankenPHP) / docker (push) Waiting to run
Static analysis / Static analysis (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, sqlite) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, sqlite) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, sqlite) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, sqlite) (push) Waiting to run
Fixes #1231.
Modify tests to account for this case.
2026-02-15 14:41:25 +01:00
Jan Böhmer
097041a43a Ran rector 2026-02-14 23:33:40 +01:00
Niklas
a355bda9da
add supplier SPN linking for BOM import (#1209)
Some checks are pending
Build assets artifact / Build assets artifact (push) Waiting to run
Docker Image Build / docker (push) Waiting to run
Docker Image Build (FrankenPHP) / docker (push) Waiting to run
Static analysis / Static analysis (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, mysql) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, postgres) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, sqlite) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, sqlite) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, sqlite) (push) Waiting to run
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, sqlite) (push) Waiting to run
* feat: add supplier SPN lookup for BOM import

Add automatic part linking via supplier part numbers (SPNs) in the
BOM importer. When a Part-DB ID is not provided, the importer now
searches for existing parts by matching supplier SPNs from the CSV
with orderdetail records in the database.

This allows automatic part linking when KiCad schematic BOMs contain
supplier information like LCSC SPN, Mouser SPN, etc., improving the
import workflow for users who track parts by supplier part numbers.

* add tests for BOM import with supplier SPN handling
2026-01-31 22:37:43 +01:00
Niklas
3aad70934b
Support dynamic supplier SPNs in BOM import comments (#1208)
* Fix: Use correct field name for LCSC supplier part numbers in BOM import

The field mapping system uses 'LCSC SPN' as the target field name for LCSC
supplier part numbers (following the pattern SupplierName + ' SPN'), but the
code in parseKiCADSchematic() was incorrectly checking for 'LCSC'.

This caused LCSC supplier part numbers to be silently ignored and not included
in the BOM entry comments during schematic import.

Changed isset($mapped_entry['LCSC']) to isset($mapped_entry['LCSC SPN']) to
match the actual field name produced by the field mapping system.

* regression test: check for LCSC SPN in comment

* Support dynamic supplier SPNs in BOM import comments

Replace hardcoded LCSC SPN handling with dynamic supplier lookup to support all configured suppliers in BOM import. This allows any supplier defined in
Part-DB to have their SPN fields recognized and included in the BOM entry
comments during BOM import.

* Optimize BOM import by only calculating supplier SPN keys once
2026-01-25 21:32:14 +01:00
Jan Böhmer
96a37a0cb0 Implemented proof of concept to convert between database types 2026-01-05 22:41:40 +01:00
Jan Böhmer
9565a9d548 Fixed error with mass creation, when elements on different level had the same name
Fixes issue #1104
2025-12-07 21:40:57 +01:00
Jan Böhmer
39ff4f81c0 Use image attachments as preview images for partkeepr imports
Fixes issue #1115
2025-12-07 20:25:39 +01:00
Jan Böhmer
c60b406157 Fixed partkeepr import with databases that do not feature custom states 2025-12-07 20:21:19 +01:00
Jan Böhmer
171508fcad Fixed deprecated csv reader usage 2025-11-30 14:34:45 +01:00
web-devinition.de
14a4f1f437
Added custom part status (#1053)
Some checks failed
Build assets artifact / Build assets artifact (push) Has been cancelled
Docker Image Build / docker (push) Has been cancelled
Docker Image Build (FrankenPHP) / docker (push) Has been cancelled
Static analysis / Static analysis (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, mysql) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, mysql) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, mysql) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, mysql) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, postgres) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, postgres) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, postgres) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, postgres) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.2, sqlite) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.3, sqlite) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.4, sqlite) (push) Has been cancelled
PHPUnit Tests / PHPUnit and coverage Test (PHP 8.5, sqlite) (push) Has been cancelled
* Benutzerdefinierten Bauteilstatus einführen

* PartCustomStateController hinzufügen

* Umstellung Migrationen bzgl. Multi-Plattform-Support.
Zunächst MySQL, SQLite Statements integrieren.

* Postgre Statements integrieren

* Semikolon in Migration entfernen

* Migration für PartCustomState aktualisieren

* Benutzerdefinierten Bauteilstatus in TableSettings aufnehmen

* PartCustomStateControllerTest: Attribute für PHPUnit-Gruppen umgestellt

* PartCustomState: Mapping für Parameter korrigieren

* PartCustomState: Darstellung und Zuordnung von Anhängen ergänzt

Die Sidebar wurde um die Anzeige des benutzerdefinierten Bauteilstatus erweitert, inklusive Vorschaubild, sofern vorhanden.

* Migrationen zusammenführen

* PartCustomState: Anpassungen bzgl. Tests

* PartCustomStateEndpoint hinzufügen

* Made custom part states plural for consistency with other entity captions

* Fixed phpunit error

* Fixed phpstan issues

---------

Co-authored-by: Marcel Diegelmann <marcel.diegelmann@gmail.com>
Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
2025-10-27 21:58:16 +01:00
Jan Böhmer
ed1e51f694 Merge branch 'feature/batch-info-provider-import' 2025-09-21 23:14:09 +02:00
barisgit
d0f2422e0d Implement functionality to import schematic csv (or any other csv for that matter), with ability to map input columns to output columns with input validation and error handling 2025-09-06 19:54:13 +02:00
Jan Böhmer
d57107ed3e Do not use ob_* functions in XSLX exporter, as this affects global state and can lead to sideffects 2025-08-31 23:05:07 +02:00
barisgit
aa29f10d51 Remove problematic tests 2025-08-31 22:15:58 +02:00
barisgit
78885ec3c5 Add more tests and fix failing ones 2025-08-31 22:15:58 +02:00
barisgit
1fb137e89f Add export functionality to batch select and fix errors 2025-08-31 22:15:58 +02:00
barisgit
facfb37383 Implement excel based import/export 2025-08-31 22:15:58 +02:00
Jan Böhmer
3fcb5ce82e Merge branch 'master' into v2 2025-08-13 16:07:20 +02:00
Jan Böhmer
58d4207b49 Fixed phpstan issues 2025-08-03 21:59:51 +02:00
Jan Böhmer
442457f11b Merge branch 'master' into settings-bundle 2025-06-15 18:39:49 +02:00
Treeed
29f92d9bd3
Split attachment paths (#848)
* fixed attachment statistics for sqlite

* Split attachment path into internal and external path, so the external source URL can be retained after a file is downloaded

* Make internal and external path for attachments nullable, to make clear that they have no internal or external path

* Added migrations for nullable columns for postgres and mysql

* Added migration for nullable internal and external pathes for sqlite

* Added translations

* Fixed upload error

* Restrict length of filename badge in attachment edit view

* Improved margins with badges in attachment edit

* Added a link to view external version from attachment edit

* Let media_url  stay in API attachments responses for backward compatibility

---------

Co-authored-by: jona <a@b.c>
Co-authored-by: Jan Böhmer <mail@jan-boehmer.de>
2025-02-22 17:29:14 +01:00
Jan Böhmer
b724b05de6 Fixed "implicitly marking parameter as nullable" deprecations in PHP 8.4 fixed 2025-02-08 00:19:20 +01:00
Jan Böhmer
8750573724 Merge branch 'master' into settings-bundle 2025-01-17 22:06:18 +01:00
Jan Böhmer
946032a101 Fixed phpstan analysis issues and bad code that showed up with phpstan 2.0 2024-12-28 23:08:08 +01:00
Jan Böhmer
5e76451d46 Try to guess the character encodings and convert it to UTF-8 on importing
This should fix issue #749
2024-11-03 22:27:24 +01:00
Jan Böhmer
2ab2b7f77d Moved default language, default timezone and base currency settings to new settings system 2024-08-03 23:14:29 +02:00
Jan Böhmer
bbf7222a6a Improved EntityExporter to handle recursive exports 2024-06-23 00:11:38 +02:00
Jan Böhmer
b7b941e3a1 Fixed error handling of structural data import
This was the reason for the exception in #632
2024-06-22 22:55:15 +02:00
Jan Böhmer
235d572f8c Use DatetimeImmutable instead of DateTime wherever possible 2024-06-22 17:36:54 +02:00
Jan Böhmer
20f32c7f12 Applied rector suggestions 2024-06-22 00:31:43 +02:00
Jan Böhmer
60325e797d Fixed the use of the removed ClassMetadataInfo constants 2024-06-09 23:45:22 +02:00
Jan Böhmer
a96c10a13e Made KiCAD bom import independent from the language of the header rows
This fixes issue #604
2024-04-29 00:09:10 +02:00
Jan Böhmer
ed4728fdb1 Correcty handle already existing elements in mass creation
This fixes issue #543
2024-03-10 00:51:37 +01:00
Jan Böhmer
d49bb1f4b5 Removed unused import statements 2024-03-03 20:34:36 +01:00
Jan Böhmer
33e36f3d2b Fixed issue with EntityImported that was caused by the changes to PartNormalizer
We now have a possibility to skip API Platforms serializer subsystem
2024-03-03 20:24:24 +01:00
Jan Böhmer
42e604245c Fixed some inspection issues 2024-03-03 19:57:31 +01:00
Jan Böhmer
fb493cc837 Fixed export of entities, if their name contained slash or other reserved charcters 2024-01-29 21:25:30 +01:00
Jan Böhmer
d9f58b935a Fixed some issues when importing parameters from partkeepr
Before values were not properly imported, if there was not a normalized version yet and units were not correctly imported
2024-01-11 00:02:32 +01:00
Jan Böhmer
301ecf6c95 Do not import prices with zero price as pricedetails during partkeepr import
These are useless placeholders by partkeepr
2024-01-10 23:40:06 +01:00
Jan Böhmer
e2146332dc Fixed problems with invalid dates during PartKeepr import with MySQL
See issue #473
2024-01-10 23:18:02 +01:00
Jan Böhmer
0af5a58dbe Renamed Storelocation entity to StorageLocation 2023-09-04 22:57:40 +02:00
Jan Böhmer
85f3ba6aaa Merge branch 'master' into api 2023-08-26 22:15:12 +02:00
Jan Böhmer
73d61f7440 Fixed PartKeepr import for storagelocation attachments
This should fix issue #334
2023-08-19 23:52:22 +02:00
Jan Böhmer
676c8eeefb Added basic API to access and edit manufacturer entities 2023-08-06 20:18:57 +02:00
Jan Böhmer
ed6b0057b7 Make the manufacturer field optional on part manufacturer data during Partkeepr import
This should fix issue #333
2023-07-26 22:55:18 +02:00
Jan Böhmer
f98e20aa84 Fixed errors importing partkeepr databases 2023-07-03 23:33:45 +02:00