Added documentation about AI features

This removes clutter from the translation panel

README.md

@@ -62,6 +62,7 @@ for the first time.
 * Automatic thumbnail generation for pictures
 * Use cloud providers (like Octopart, Digikey, Farnell, LCSC or TME) to automatically get part information, datasheets, and
   prices for parts
+* Retrieve part information from arbitrary shop websites, using either conventional data extraction from structured metadata, or AI based data extraction
 * API to access Part-DB from other applications/scripts
 * [Integration with KiCad](https://docs.part-db.de/usage/eda_integration.html): Use Part-DB as the central datasource for your
   KiCad and see available parts from Part-DB directly inside KiCad.

docs/index.md

@@ -47,6 +47,7 @@ It is installed on a web server and so can be accessed with any browser without
 * Easy migration from an existing PartKeepr instance (see [here]({%link partkeepr_migration.md %}))
 * Use cloud providers (like Octopart, Digikey, Farnell, Mouser, or TME) to automatically get part information, datasheets, and
   prices for parts (see [here]({% link usage/information_provider_system.md %}))
+* Retrieve part information from arbitrary shop websites, using either conventional data extraction from structured metadata, or AI based data extraction
 * API to access Part-DB from other applications/scripts
 * [Integration with KiCad]({%link usage/eda_integration.md %}): Use Part-DB as the central datasource for your
   KiCad and see available parts from Part-DB directly inside KiCad.

docs/usage/ai.md

@@ -0,0 +1,27 @@
+---
+layout: default
+title: AI features
+nav_order: 6
+parent: Usage
+---
+
+# AI features
+
+Part-DB can utilize large language Models (LLMs) to provide AI-powered features that can assist you in managing your parts and projects.
+For now this is mostly the ability to extract part information from websites without any structured data.
+
+## AI platforms
+
+Part-DB is platform agnostic and can work with different AI platforms, both locally and in the cloud. They can be configured in the "AI" tab in the system settings.
+Currently, the following platforms are supported:
+
+### OpenRouter
+
+[OpenRouter](https://openrouter.ai/) is a platform that provides access to various LLMs, including models from OpenAI, Anthropic, and more. 
+You can use OpenRouter to connect to different LLMs and use them for Part-DB's AI features.
+You need to supply an API key for OpenRouter to use it as an AI platform in Part-DB.
+
+### LMStudio
+
+[LMStudio](https://lmstudio.ai/) is a local LLM hosting solution that allows you to run LLMs on your own hardware. You can use LMStudio to host your own LLM and connect it to Part-DB for AI features.
+Currently only LMStudio without any authentication is supported. Supply your LMStudio instance URL (including the port) to use it as an AI platform in Part-DB.

docs/usage/information_provider_system.md

@@ -111,6 +111,19 @@ may have privacy and security implications.
 Following env configuration options are available:
 * `PROVIDER_GENERIC_WEB_ENABLED`: Set this to `1` to enable the Generic Web URL Provider (optional, default: `0`)
 
+### AI Web Extractor
+The AI web extractor provider can extract part information from any webpage using AI-based techniques. It is designed to handle unstructured data and can extract relevant information even from websites that do not use structured data formats like Schema.org. 
+This provider can be particularly useful for extracting information from websites that have complex layouts or do not follow standard e-commerce practices.
+It also potentially extracts more detailed information than the Generic Web URL Provider, as it is not limited to the fields defined in the Schema.org format.
+
+To use the AI Web Extractor, you need to setup an AI platform, in the AI settings tab, and chose a model, which support structured output.
+For many use cases a small and cheap model like `google/gemini-2.5-flash-lite` will be sufficient, coming down to costs like 0.003$ per request.
+For more complex websites, or if you wanna use the LLM for translation purposes too, you should consider a more powerful model.
+
+You can add some additional instructions for the model, which gets added to the system prompt, to tweak the output of the model.
+
+The provider will download the HTML of the given URL, convert it to markdown and send it to the LLM toghether with structured data extracted from the webpage via conventional methods.
+
 ### Octopart
 
 The Octopart provider uses the [Octopart / Nexar API](https://nexar.com/api) to search for parts and get information.

src/Controller/InfoProviderController.php

@@ -26,8 +26,10 @@ namespace App\Controller;
 use App\Entity\Parts\Manufacturer;
 use App\Entity\Parts\Part;
 use App\Exceptions\OAuthReconnectRequiredException;
+use App\Form\InfoProviderSystem\FromURLFormType;
 use App\Form\InfoProviderSystem\PartSearchType;
 use App\Services\InfoProviderSystem\ExistingPartFinder;
+use App\Services\InfoProviderSystem\CreateFromUrlHelper;
 use App\Services\InfoProviderSystem\PartInfoRetriever;
 use App\Services\InfoProviderSystem\ProviderRegistry;
 use App\Services\InfoProviderSystem\Providers\GenericWebProvider;
@@ -219,35 +221,35 @@ class InfoProviderController extends  AbstractController
     }
 
     #[Route('/from_url', name: 'info_providers_from_url')]
-    public function fromURL(Request $request, GenericWebProvider $provider): Response
+    public function fromURL(Request $request, GenericWebProvider $provider, CreateFromUrlHelper $fromUrlHelper): Response
     {
         $this->denyAccessUnlessGranted('@info_providers.create_parts');
 
-        if (!$provider->isActive()) {
+        if (!$fromUrlHelper->canCreateFromUrl()) {
             $this->addFlash('error', "Generic Web Provider is not active. Please enable it in the provider settings.");
             return $this->redirectToRoute('info_providers_list');
         }
 
-        $formBuilder = $this->createFormBuilder();
-        $formBuilder->add('url', UrlType::class, [
-            'label' => 'info_providers.from_url.url.label',
-            'required' => true,
-        ]);
-        $formBuilder->add('submit', SubmitType::class, [
-            'label' => 'info_providers.search.submit',
-        ]);
-
-        $form = $formBuilder->getForm();
+        $form = $this->createForm(FromURLFormType::class);
         $form->handleRequest($request);
 
         $partDetail = null;
         if ($form->isSubmitted() && $form->isValid()) {
             //Try to retrieve the part detail from the given URL
             $url = $form->get('url')->getData();
+
+            $method = $form->get('method')->getData();
+            $no_cache = $form->get('no_cache')->getData();
+            $skip_delegation = $form->get('skip_delegation')->getData();
+
             try {
+                //It's okay if we use the cached results here, as its just for convenience
                 $searchResult = $this->infoRetriever->searchByKeyword(
                     keyword: $url,
-                    providers: [$provider]
+                    providers: [$method],
+                    options: [
+                        InfoProviderInterface::OPTION_SKIP_DELEGATION => $skip_delegation,
+                    ]
                 );
 
                 if (count($searchResult) === 0) {
@@ -258,6 +260,8 @@ class InfoProviderController extends  AbstractController
                     return $this->redirectToRoute('info_providers_create_part', [
                         'providerKey' => $searchResult->provider_key,
                         'providerId' => $searchResult->provider_id,
+                        'no_cache' => $no_cache ? 1 : null,
+                        'skip_delegation' => $skip_delegation ? 1 : null,
                     ]);
                 }
             } catch (ExceptionInterface $e) {

src/Controller/PartController.php

@@ -286,8 +286,12 @@ final class PartController extends AbstractController
 
         //Force info providers to not use cache, when retrieving part details for creating a new part, because otherwise we might end up with outdated information
         $no_cache = $request->query->getBoolean('no_cache', false);
+        $skip_delegation = $request->query->getBoolean('skip_delegation', false);
 
-        $dto = $infoRetriever->getDetails($providerKey, $providerId, [InfoProviderInterface::OPTION_NO_CACHE => $no_cache]);
+        $dto = $infoRetriever->getDetails($providerKey, $providerId, [
+            InfoProviderInterface::OPTION_NO_CACHE => $no_cache,
+            InfoProviderInterface::OPTION_SKIP_DELEGATION => $skip_delegation,
+        ]);
         $new_part = $infoRetriever->dtoToPart($dto);
 
         if ($new_part->getCategory() === null || $new_part->getCategory()->getID() === null) {

src/Controller/TypeaheadController.php

@@ -244,7 +244,7 @@ class TypeaheadController extends AbstractController
 
         $capability_filter = $request->query->getEnum('capability', Capability::class);
 
-        $models = $cache->get('ai_models_'.$platform->value.'_'.($capability_filter?->value ?? 'all'),
+        $models = $cache->get('ai_models_'.$platform->value.'_'.($capability_filter->value ?? 'all'),
             function (ItemInterface $item) use ($platformRegistry, $platform, $capability_filter) {
                 $item->expiresAfter(3600); //Cache for 1 hour
                 if ($capability_filter === null) {
@@ -253,7 +253,7 @@ class TypeaheadController extends AbstractController
 
                 //Otherwise filter the models by the capability
                 return array_filter($platformRegistry->getPlatform($platform)->getModelCatalog()->getModels(),
-                    static fn(array $model) => in_array($capability_filter, $model['capabilities'] ?? [], true)
+                    static fn(array $model) => in_array($capability_filter, $model['capabilities'], true)
                 );
             });
 

src/Form/InfoProviderSystem/FromURLFormType.php

@@ -0,0 +1,87 @@
+<?php
+/*
+ * This file is part of Part-DB (https://github.com/Part-DB/Part-DB-symfony).
+ *
+ *  Copyright (C) 2019 - 2026 Jan Böhmer (https://github.com/jbtronics)
+ *
+ *  This program is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU Affero General Public License as published
+ *  by the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU Affero General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Affero General Public License
+ *  along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+declare(strict_types=1);
+
+
+namespace App\Form\InfoProviderSystem;
+
+use App\Services\InfoProviderSystem\ProviderRegistry;
+use Symfony\Component\Form\AbstractType;
+use Symfony\Component\Form\Extension\Core\Type\CheckboxType;
+use Symfony\Component\Form\Extension\Core\Type\ChoiceType;
+use Symfony\Component\Form\Extension\Core\Type\SubmitType;
+use Symfony\Component\Form\Extension\Core\Type\UrlType;
+use Symfony\Component\Form\FormBuilderInterface;
+
+class FromURLFormType extends AbstractType
+{
+    public function __construct(private readonly ProviderRegistry $providerRegistry)
+    {
+
+    }
+
+    public function buildForm(FormBuilderInterface $builder, array $options): void
+    {
+        $builder->add('url', UrlType::class, [
+            'label' => 'info_providers.from_url.url.label',
+            'required' => true,
+        ]);
+
+
+        $builder->add('method', ChoiceType::class, [
+            'expanded' => true,
+            'data' => 'generic_web', //Default value
+            'label' => 'info_providers.from_url.method',
+            'choices' => [
+                'info_providers.from_url.method.generic_web' => 'generic_web',
+                'info_providers.from_url.method.ai_web' => 'ai_web',
+            ],
+            'choice_attr' => function ($choice, $key, $value) {
+                //Disable all providers that are not active
+                $provider = $this->providerRegistry->getProviderByKey($value);
+                if (!$provider->isActive()) {
+                    return ['disabled' => 'disabled'];
+                }
+
+                return [];
+            },
+
+            //Render the choices as inline radio buttons
+            'label_attr' => [
+                'class' => 'radio-inline',
+            ],
+        ]);
+
+        $builder->add('no_cache', CheckboxType::class, [
+            'label' => 'info_providers.from_url.no_cache',
+            'required' => false,
+        ]);
+
+        $builder->add('skip_delegation', CheckboxType::class, [
+            'label' => 'info_providers.from_url.skip_delegation',
+            'required' => false,
+        ]);
+
+        $builder->add('submit', SubmitType::class, [
+            'label' => 'info_providers.search.submit',
+        ]);
+    }
+}

src/Form/Settings/AiPlatformChoiceType.php

@@ -41,7 +41,7 @@ final class AiPlatformChoiceType extends AbstractType
     {
     }
 
-    public function getParent(): ?string
+    public function getParent(): string
     {
         return EnumType::class;
     }

src/Services/AI/AIPlatformRegistry.php

@@ -43,7 +43,6 @@ final readonly class AIPlatformRegistry
 
     public function __construct(
         SettingsManagerInterface $settingsManager,
-
         #[AutowireIterator(tag: 'ai.platform', indexAttribute: 'name')]
         iterable $platforms,
     ) {

src/Services/AI/AIPlatforms.php

@@ -52,8 +52,6 @@ enum AIPlatforms: string implements TranslatableInterface
         return match ($this) {
             self::LMSTUDIO => LMStudioSettings::class,
             self::OPENROUTER => OpenRouterSettings::class,
-
-            default => throw new \InvalidArgumentException(sprintf('No settings class defined for AI platform "%s".', $this->name)),
         };
     }
 

src/Services/AI/AcceptAllModelsCatalog.php

@@ -34,6 +34,7 @@ use Symfony\Component\DependencyInjection\Attribute\AsDecorator;
  * This is a workaround for outdated/incomplete model catalogs provided by AI platforms, which do not contain all available models, or do not update their catalogs frequently enough.
  */
 #[AsDecorator('ai.platform.model_catalog.lmstudio')]
+#[AsDecorator('ai.platform.model_catalog.openrouter')]
 final readonly class AcceptAllModelsCatalog implements ModelCatalogInterface
 {
 

src/Services/InfoProviderSystem/CreateFromUrlHelper.php

@@ -0,0 +1,109 @@
+<?php
+/*
+ * This file is part of Part-DB (https://github.com/Part-DB/Part-DB-symfony).
+ *
+ *  Copyright (C) 2019 - 2026 Jan Böhmer (https://github.com/jbtronics)
+ *
+ *  This program is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU Affero General Public License as published
+ *  by the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU Affero General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Affero General Public License
+ *  along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+declare(strict_types=1);
+
+
+namespace App\Services\InfoProviderSystem;
+
+use App\Entity\UserSystem\User;
+use App\Exceptions\ProviderIDNotSupportedException;
+use App\Services\InfoProviderSystem\DTOs\PartDetailDTO;
+use App\Services\InfoProviderSystem\DTOs\SearchResultDTO;
+use App\Services\InfoProviderSystem\Providers\InfoProviderInterface;
+use Symfony\Bundle\SecurityBundle\Security;
+
+final readonly class CreateFromUrlHelper
+{
+    public function __construct(private Security $security,
+        private ProviderRegistry $providerRegistry,
+        private PartInfoRetriever $infoRetriever,
+    )
+    {
+    }
+
+    /**
+     * Checks if at least one provider can create parts from an URL and the current user is allowed to use it.
+     * This is used to determine if the "From URL" feature should be shown to the user.
+     * @return bool
+     */
+    public function canCreateFromUrl(): bool
+    {
+        if (!$this->security->isGranted('@info_providers.create_parts')) {
+            return false;
+        }
+
+        //Check if either the generic web provider or the ai web provider is active
+        $genericWebProvider = $this->providerRegistry->getProviderByKey('generic_web');
+        $aiWebProvider = $this->providerRegistry->getProviderByKey('ai_web');
+
+        return $genericWebProvider->isActive() || $aiWebProvider->isActive();
+    }
+
+    /**
+     * Delegates the URL to another provider if possible, otherwise return null
+     * @param  string  $url
+     * @return SearchResultDTO|null
+     */
+    public function delegateToOtherProvider(string $url, InfoProviderInterface $callingInfoProvider): ?SearchResultDTO
+    {
+        //Extract domain from url:
+        $host = parse_url($url, PHP_URL_HOST);
+        if ($host === false || $host === null) {
+            return null;
+        }
+
+        $provider = $this->providerRegistry->getProviderHandlingDomain($host);
+
+        if ($provider !== null && $provider->isActive() && $provider->getProviderKey() !== $callingInfoProvider->getProviderKey()) {
+            try {
+                $id = $provider->getIDFromURL($url);
+                if ($id !== null) {
+                    $results = $this->infoRetriever->searchByKeyword($id, [$provider]);
+                    if (count($results) > 0) {
+                        return $results[0];
+                    }
+                }
+                return null;
+            } catch (ProviderIDNotSupportedException $e) {
+                //Ignore and continue
+                return null;
+            }
+        }
+
+        return null;
+    }
+
+    /**
+     * Delegates the URL to another provider if possible and returns the details, otherwise return null
+     * @param  string  $url
+     * @param  InfoProviderInterface  $callingInfoProvider
+     * @return PartDetailDTO|null
+     */
+    public function delegateToOtherProviderDetails(string $url, InfoProviderInterface $callingInfoProvider): ?PartDetailDTO
+    {
+        $delegatedResult = $this->delegateToOtherProvider($url, $callingInfoProvider);
+        if ($delegatedResult !== null) {
+            return $this->infoRetriever->getDetailsForSearchResult($delegatedResult);
+        }
+
+        return null;
+    }
+}

src/Services/InfoProviderSystem/Providers/AIWebProvider.php

@@ -27,12 +27,14 @@ namespace App\Services\InfoProviderSystem\Providers;
 use App\Exceptions\ProviderIDNotSupportedException;
 use App\Helpers\RandomizeUseragentHttpClient;
 use App\Services\AI\AIPlatformRegistry;
+use App\Services\InfoProviderSystem\CreateFromUrlHelper;
 use App\Services\InfoProviderSystem\DTOJsonSchemaConverter;
 use App\Services\InfoProviderSystem\DTOs\PartDetailDTO;
 use App\Settings\InfoProviderSystem\AIExtractorSettings;
 use Brick\Schema\SchemaReader;
 use Jkphl\Micrometa;
 use League\HTMLToMarkdown\HtmlConverter;
+use Psr\Cache\CacheItemPoolInterface;
 use Symfony\AI\Platform\Message\Message;
 use Symfony\AI\Platform\Message\MessageBag;
 use Symfony\Component\DomCrawler\Crawler;
@@ -43,11 +45,11 @@ use Symfony\Contracts\HttpClient\HttpClientInterface;
 use function Symfony\Component\String\u;
 
 
-final class AIInfoExtractor implements InfoProviderInterface
+final class AIWebProvider implements InfoProviderInterface
 {
     use FixAndValidateUrlTrait;
 
-    private const DISTRIBUTOR_NAME = 'AI Extracted';
+    private const DISTRIBUTOR_NAME = 'Website';
 
     private readonly HttpClientInterface $httpClient;
 
@@ -56,6 +58,8 @@ final class AIInfoExtractor implements InfoProviderInterface
         private readonly AIExtractorSettings $settings,
         private readonly AIPlatformRegistry $AIPlatformRegistry,
         private readonly DTOJsonSchemaConverter $jsonSchemaConverter,
+        private readonly CacheItemPoolInterface $partInfoCache,
+        private readonly CreateFromUrlHelper $createFromUrlHelper,
     ) {
         //Use NoPrivateNetworkHttpClient to prevent SSRF vulnerabilities, and RandomizeUseragentHttpClient to make it harder for servers to block us
         $this->httpClient = (new RandomizeUseragentHttpClient(new NoPrivateNetworkHttpClient($httpClient)))->withOptions(
@@ -68,17 +72,17 @@ final class AIInfoExtractor implements InfoProviderInterface
     public function getProviderInfo(): array
     {
         return [
-            'name' => 'AI Information Extractor',
-            'description' => 'Extract part info from any URL using OpenRouter LLM',
+            'name' => 'AI Web Extractor',
+            'description' => 'Extract part info from any URL using LLM',
             //'url' => 'https://openrouter.ai',
-            'disabled_help' => 'Configure OpenRouter API key in settings',
+            'disabled_help' => 'Configure AI settings',
             'settings_class' => AIExtractorSettings::class,
         ];
     }
 
     public function getProviderKey(): string
     {
-        return 'ai_extractor';
+        return 'ai_web';
     }
 
     public function isActive(): bool
@@ -88,9 +92,23 @@ final class AIInfoExtractor implements InfoProviderInterface
 
     public function searchByKeyword(string $keyword, array $options = []): array
     {
+        $url = $this->fixAndValidateURL($keyword);
+
+        if (!($options[self::OPTION_SKIP_DELEGATION] ?? false)) {
+            //Before loading the page, try to delegate to another provider
+            $delegatedPart = $this->createFromUrlHelper->delegateToOtherProvider($url, $this);
+            if ($delegatedPart !== null) {
+                return [$delegatedPart];
+            }
+        }
+
         try {
+
+            $new_options = $options;
+            $new_options[self::OPTION_SKIP_DELEGATION] = true; //Skip delegation for the getDetails call to prevent infinite loops
+
             return [
-                $this->getDetails($keyword)
+                $this->getDetails($keyword, $new_options)
             ]; } catch (ProviderIDNotSupportedException $e) {
             return [];
         }
@@ -100,16 +118,32 @@ final class AIInfoExtractor implements InfoProviderInterface
     {
         $url = $this->fixAndValidateURL($id);
 
+        if (!($options[self::OPTION_SKIP_DELEGATION] ?? false)) {
+            //Before loading the page, try to delegate to another provider
+            $delegatedPart = $this->createFromUrlHelper->delegateToOtherProviderDetails($url, $this);
+            if ($delegatedPart !== null) {
+                return $delegatedPart;
+            }
+        }
+
+        //Check if we have a cached result for this URL, to avoid unnecessary LLM calls, which can be slow and costly.
+        $cacheKey = 'ai_web_'.hash('xxh3', $url);
+
+        //If ignore cache option is set, skip cache and fetch fresh data
+        if ($options[self::OPTION_NO_CACHE] ?? false) {
+            $this->partInfoCache->deleteItem($cacheKey);
+        }
+
+        //Return cached result if available
+        $cacheItem = $this->partInfoCache->getItem($cacheKey);
+        if ($cacheItem->isHit()) {
+            return $cacheItem->get();
+        }
+
         // Fetch HTML content
         $response = $this->httpClient->request('GET', $url);
         $html = $response->getContent();
 
-        // Clean HTML
-        /*$cleanedHtml = $this->cleanHTML($html);
-
-        // Truncate to max content length
-        $truncatedHtml = $this->truncateHTML($cleanedHtml, $this->settings->maxContentLength);*/
-
         //Convert html to markdown, to provide a cleaner input to the LLM.
         $markdown = $this->htmlToMarkdown($html);
         //Truncate markdown to max content length, if needed
@@ -124,6 +158,11 @@ final class AIInfoExtractor implements InfoProviderInterface
         // Build and return PartDetailDTO
         $result = $this->jsonSchemaConverter->jsonToDTO($llmResponse, $this->getProviderKey(), $url, $url, self::DISTRIBUTOR_NAME);
 
+        // Cache the result for future use, to improve performance and reduce costs.
+        $cacheItem->set($result);
+        $cacheItem->expiresAfter(3600 * 2); //Cache for 2 hours, as web content can change frequently, but we still want to benefit from caching for repeated accesses.
+        $this->partInfoCache->save($cacheItem);
+
         return $result;
     }
 

src/Services/InfoProviderSystem/Providers/GenericWebProvider.php

@@ -25,6 +25,7 @@ namespace App\Services\InfoProviderSystem\Providers;
 
 use App\Exceptions\ProviderIDNotSupportedException;
 use App\Helpers\RandomizeUseragentHttpClient;
+use App\Services\InfoProviderSystem\CreateFromUrlHelper;
 use App\Services\InfoProviderSystem\DTOs\ParameterDTO;
 use App\Services\InfoProviderSystem\DTOs\PartDetailDTO;
 use App\Services\InfoProviderSystem\DTOs\PriceDTO;
@@ -50,14 +51,12 @@ class GenericWebProvider implements InfoProviderInterface
 
     use FixAndValidateUrlTrait;
 
-    public const OPTION_CHECK_FOR_DELEGATION = 'check_for_delegation';
-
     public const DISTRIBUTOR_NAME = 'Website';
 
     private readonly HttpClientInterface $httpClient;
 
     public function __construct(HttpClientInterface $httpClient, private readonly GenericWebProviderSettings $settings,
-        private readonly ProviderRegistry $providerRegistry, private readonly PartInfoRetriever $infoRetriever,
+        private readonly CreateFromUrlHelper $createFromUrlHelper,
     )
     {
         //Use NoPrivateNetworkHttpClient to prevent SSRF vulnerabilities, and RandomizeUseragentHttpClient to make it harder for servers to block us
@@ -93,15 +92,19 @@ class GenericWebProvider implements InfoProviderInterface
     {
         $url = $this->fixAndValidateURL($keyword);
 
-        //Before loading the page, try to delegate to another provider
-        $delegatedPart = $this->delegateToOtherProvider($url);
-        if ($delegatedPart !== null) {
-            return [$delegatedPart];
+        if (!($options[self::OPTION_SKIP_DELEGATION] ?? false)) {
+            //Before loading the page, try to delegate to another provider
+            $delegatedPart = $this->createFromUrlHelper->delegateToOtherProvider($url, $this);
+            if ($delegatedPart !== null) {
+                return [$delegatedPart];
+            }
         }
 
         try {
+            $new_options = $options;
+            $new_options[self::OPTION_SKIP_DELEGATION] = true; //Skip delegation for the getDetails call to prevent infinite loops
             return [
-                $this->getDetails($keyword, [self::OPTION_CHECK_FOR_DELEGATION => false]) //We already tried delegation
+                $this->getDetails($keyword, $new_options)
             ]; } catch (ProviderIDNotSupportedException $e) {
             return [];
         }
@@ -278,53 +281,16 @@ class GenericWebProvider implements InfoProviderInterface
         return null;
     }
 
-    /**
-     * Delegates the URL to another provider if possible, otherwise return null
-     * @param  string  $url
-     * @return SearchResultDTO|null
-     */
-    private function delegateToOtherProvider(string $url): ?SearchResultDTO
-    {
-        //Extract domain from url:
-        $host = parse_url($url, PHP_URL_HOST);
-        if ($host === false || $host === null) {
-            return null;
-        }
-
-        $provider = $this->providerRegistry->getProviderHandlingDomain($host);
-
-        if ($provider !== null && $provider->isActive() && $provider->getProviderKey() !== $this->getProviderKey()) {
-            try {
-                $id = $provider->getIDFromURL($url);
-                if ($id !== null) {
-                    $results = $this->infoRetriever->searchByKeyword($id, [$provider]);
-                    if (count($results) > 0) {
-                        return $results[0];
-                    }
-                }
-                return null;
-            } catch (ProviderIDNotSupportedException $e) {
-                //Ignore and continue
-                return null;
-            }
-        }
-
-        return null;
-    }
-
-
 
     public function getDetails(string $id, array $options = []): PartDetailDTO
     {
-        //We check for delegation by default
-        $check_for_delegation = $options[self::OPTION_CHECK_FOR_DELEGATION] ?? true;
         $url = $this->fixAndValidateURL($id);
 
-        if ($check_for_delegation) {
+        if (!($options[self::OPTION_SKIP_DELEGATION] ?? false)) {
             //Before loading the page, try to delegate to another provider
-            $delegatedPart = $this->delegateToOtherProvider($url);
+            $delegatedPart = $this->createFromUrlHelper->delegateToOtherProviderDetails($url, $this);
             if ($delegatedPart !== null) {
-                return $this->infoRetriever->getDetailsForSearchResult($delegatedPart);
+                return $delegatedPart;
             }
         }
 

src/Services/InfoProviderSystem/Providers/InfoProviderInterface.php

@@ -29,6 +29,7 @@ use App\Services\InfoProviderSystem\DTOs\SearchResultDTO;
 interface InfoProviderInterface
 {
     public const OPTION_NO_CACHE = 'no_cache'; // if set to true, the provider should not use any cache and retrieve fresh data from the source
+    public const OPTION_SKIP_DELEGATION = 'skip_delegation'; // if set to true, the provider should not delegate the request to other providers, even if it supports delegation.
 
     /**
      * Get information about this provider

src/Settings/AISettings/LMStudioSettings.php

@@ -31,6 +31,7 @@ use Jbtronics\SettingsBundle\Settings\Settings;
 use Jbtronics\SettingsBundle\Settings\SettingsParameter;
 use Jbtronics\SettingsBundle\Settings\SettingsTrait;
 use Symfony\Component\Form\Extension\Core\Type\UrlType;
+use Symfony\Component\Translation\StaticMessage;
 use Symfony\Component\Translation\TranslatableMessage as TM;
 
 #[Settings(name: 'ai_lmstudio', label: new TM("settings.ai.lmstudio"))]
@@ -41,6 +42,7 @@ class LMStudioSettings implements AIPlatformSettingsInterface
 
     #[SettingsParameter(label: new TM("settings.ai.lmstudio.hosturl"),
         formType: UrlType::class,
+        formOptions: ["attr" => ["placeholder" =>  new StaticMessage("http://localhost:1234")]],
         envVar: "AI_LMSTUDIO_HOSTURL", envVarMode: EnvVarMode::OVERWRITE)]
     public ?string $hostURL = null;
 

src/Settings/InfoProviderSystem/CanopySettings.php

@@ -72,7 +72,7 @@ class CanopySettings
     /**
      * @var string The domain used internally for the API requests. This is not necessarily the same as the domain shown to the user, which is determined by the keys of the ALLOWED_DOMAINS constant
      */
-    #[SettingsParameter(label: new TM("settings.ips.tme.country"), formType: ChoiceType::class, formOptions: ["choices" => self::ALLOWED_DOMAINS])]
+    #[SettingsParameter(label: new TM("settings.ips.tme.country"), formType: ChoiceType::class, formOptions: ["choices" => self::ALLOWED_DOMAINS, 'translation_domain' => false])]
     public string $domain = "DE";
 
     /**

src/Twig/MiscExtension.php

@@ -22,6 +22,7 @@ declare(strict_types=1);
  */
 namespace App\Twig;
 
+use App\Services\InfoProviderSystem\CreateFromUrlHelper;
 use Twig\Attribute\AsTwigFunction;
 use App\Settings\SettingsIcon;
 use Symfony\Component\HttpFoundation\Request;
@@ -34,7 +35,7 @@ use Twig\Extension\AbstractExtension;
 
 final readonly class MiscExtension
 {
-    public function __construct(private EventCommentNeededHelper $eventCommentNeededHelper)
+    public function __construct(private EventCommentNeededHelper $eventCommentNeededHelper, private CreateFromUrlHelper $fromUrlHelper)
     {
     }
 
@@ -84,4 +85,14 @@ final readonly class MiscExtension
 
         return $request->getBaseUrl().$request->getPathInfo().$qs;
     }
+
+    /**
+     * Returns true if the from url provider is active, false otherwise.
+     * @return bool
+     */
+    #[AsTwigFunction(name: 'create_from_url_active')]
+    public function create_from_url_active(): bool
+    {
+        return $this->fromUrlHelper->canCreateFromUrl();
+    }
 }

templates/_navbar.html.twig

@@ -52,7 +52,7 @@
                                             {% trans %}info_providers.search.title{% endtrans %}
                                         </a>
                                     </li>
-                                    {% if settings_instance('generic_web_provider').enabled %}
+                                    {% if create_from_url_active() %}
                                         <li>
                                             <a class="dropdown-item" href="{{ path('info_providers_from_url') }}">
                                                 <i class="fa-fw fa-solid fa-book-atlas"></i>

templates/info_providers/from_url/from_url.html.twig

@@ -16,6 +16,22 @@
 
     {{ form_start(form) }}
     {{ form_row(form.url) }}
+
+    {{ form_row(form.method) }}
+
+    <div class="row mb-2">
+        <div class="{{ col_input }} {{ offset_label }}">
+            <a data-bs-toggle="collapse" href="#infoSearchAdvancedPanel">{% trans %}info_providers.search.advanced_options{% endtrans %}</a>
+        </div>
+    </div>
+
+    <div class="collapse" id="infoSearchAdvancedPanel">
+        <div class="card card-body mb-2">
+            {{ form_row(form.no_cache) }}
+            {{ form_row(form.skip_delegation) }}
+        </div>
+    </div>
+
     {{ form_row(form.submit) }}
     {{ form_end(form) }}
 {% endblock %}

tests/Services/AI/AIPlatformRegistryTest.php

@@ -0,0 +1,99 @@
+<?php
+/*
+ * This file is part of Part-DB (https://github.com/Part-DB/Part-DB-symfony).
+ *
+ *  Copyright (C) 2019 - 2026 Jan Böhmer (https://github.com/jbtronics)
+ *
+ *  This program is free software: you can redistribute it and/or modify
+ *  it under the terms of the GNU Affero General Public License as published
+ *  by the Free Software Foundation, either version 3 of the License, or
+ *  (at your option) any later version.
+ *
+ *  This program is distributed in the hope that it will be useful,
+ *  but WITHOUT ANY WARRANTY; without even the implied warranty of
+ *  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+ *  GNU Affero General Public License for more details.
+ *
+ *  You should have received a copy of the GNU Affero General Public License
+ *  along with this program.  If not, see <https://www.gnu.org/licenses/>.
+ */
+
+/**
+ * Tests for App\Services\AI\AIPlatformRegistry
+ */
+declare(strict_types=1);
+
+namespace App\Tests\Services\AI;
+
+use App\Services\AI\AIPlatformRegistry;
+use App\Services\AI\AIPlatforms;
+use App\Services\AI\AIPlatformSettingsInterface;
+use Jbtronics\SettingsBundle\Manager\SettingsManagerInterface;
+use PHPUnit\Framework\TestCase;
+use Symfony\AI\Platform\PlatformInterface;
+
+class AIPlatformRegistryTest extends TestCase
+{
+    public function testRegistersEnabledPlatformsAndReturnsPlatform(): void
+    {
+        // Create a platform mock and expose it under the service tag name (openrouter)
+        $platformMock = $this->createMock(PlatformInterface::class);
+
+        // Settings for OpenRouter -> enabled
+        $openRouterSettings = $this->createMock(AIPlatformSettingsInterface::class);
+        $openRouterSettings->method('isAIPlatformEnabled')->willReturn(true);
+
+        // Settings for LMStudio -> disabled
+        $lmSettings = $this->createMock(AIPlatformSettingsInterface::class);
+        $lmSettings->method('isAIPlatformEnabled')->willReturn(false);
+
+        // Settings manager should return the corresponding settings object depending on the requested class name
+        $settingsManager = $this->createMock(SettingsManagerInterface::class);
+        $settingsManager->method('get')->willReturnMap([
+            [AIPlatforms::OPENROUTER->toSettingsClass(), $openRouterSettings],
+            [AIPlatforms::LMSTUDIO->toSettingsClass(), $lmSettings],
+        ]);
+
+        $platforms = new \ArrayIterator([
+            AIPlatforms::OPENROUTER->toServiceTagName() => $platformMock,
+        ]);
+
+        $registry = new AIPlatformRegistry($settingsManager, $platforms);
+
+        // OPENROUTER should be enabled and retrievable
+        $this->assertTrue($registry->isEnabled(AIPlatforms::OPENROUTER));
+        $this->assertSame($platformMock, $registry->getPlatform(AIPlatforms::OPENROUTER));
+
+        // LMSTUDIO is either not registered or disabled -> should not be enabled
+        $this->assertFalse($registry->isEnabled(AIPlatforms::LMSTUDIO));
+        $this->expectException(\InvalidArgumentException::class);
+        $registry->getPlatform(AIPlatforms::LMSTUDIO);
+    }
+
+    public function testGetEnabledPlatformsReturnsIndexedArray(): void
+    {
+        $platformMock = $this->createMock(PlatformInterface::class);
+
+        $openRouterSettings = $this->createMock(AIPlatformSettingsInterface::class);
+        $openRouterSettings->method('isAIPlatformEnabled')->willReturn(true);
+
+        $settingsManager = $this->createMock(SettingsManagerInterface::class);
+        $settingsManager->method('get')->willReturnMap([
+            [AIPlatforms::OPENROUTER->toSettingsClass(), $openRouterSettings],
+            [AIPlatforms::LMSTUDIO->toSettingsClass(), $this->createMock(AIPlatformSettingsInterface::class)],
+        ]);
+
+        $platforms = new \ArrayIterator([
+            AIPlatforms::OPENROUTER->toServiceTagName() => $platformMock,
+            // lmstudio not registered
+        ]);
+
+        $registry = new AIPlatformRegistry($settingsManager, $platforms);
+
+        $enabled = $registry->getEnabledPlatforms();
+
+        $this->assertArrayHasKey(AIPlatforms::OPENROUTER->value, $enabled);
+        $this->assertSame($platformMock, $enabled[AIPlatforms::OPENROUTER->value]);
+    }
+}
+

translations/messages.en.xlf

@@ -13157,5 +13157,59 @@ Buerklin-API Authentication server:
         <target>Do not cache result details / Force fresh part detail retrieval</target>
       </segment>
     </unit>
+    <unit id="Ja8CCDb" name="info_providers.from_url.method.generic_web">
+      <segment>
+        <source>info_providers.from_url.method.generic_web</source>
+        <target>Classic Web Scraper</target>
+      </segment>
+    </unit>
+    <unit id="s0kUzFW" name="info_providers.from_url.method.ai_web">
+      <segment>
+        <source>info_providers.from_url.method.ai_web</source>
+        <target>AI Web Scraper</target>
+      </segment>
+    </unit>
+    <unit id="HEFB1OC" name="info_providers.from_url.method">
+      <segment>
+        <source>info_providers.from_url.method</source>
+        <target>Method</target>
+      </segment>
+    </unit>
+    <unit id="9pG0VtU" name="info_providers.from_url.no_cache">
+      <segment>
+        <source>info_providers.from_url.no_cache</source>
+        <target>Ignore cache / Force fresh info retrieval</target>
+      </segment>
+    </unit>
+    <unit id="302Jgvm" name="info_providers.from_url.skip_delegation">
+      <segment>
+        <source>info_providers.from_url.skip_delegation</source>
+        <target>Do not delegate to specialized info providers</target>
+      </segment>
+    </unit>
+    <unit id="pruvlK8" name="settings.ips.ai_extractor">
+      <segment>
+        <source>settings.ips.ai_extractor</source>
+        <target>AI Web Extractor</target>
+      </segment>
+    </unit>
+    <unit id="wILs7pS" name="settings.ips.ai_extractor.description">
+      <segment>
+        <source>settings.ips.ai_extractor.description</source>
+        <target>This info provider uses an large language model (LLM) to extract detailed part information from arbitary shop URLs.</target>
+      </segment>
+    </unit>
+    <unit id="1ShZ.1i" name="settings.ai.openrouter.help">
+      <segment>
+        <source>settings.ai.openrouter.help</source>
+        <target>Access to many AI models via openrouter.ai</target>
+      </segment>
+    </unit>
+    <unit id="kRcgIf0" name="settings.ai.lmstudio.hosturl">
+      <segment>
+        <source>settings.ai.lmstudio.hosturl</source>
+        <target>Host URL</target>
+      </segment>
+    </unit>
   </file>
 </xliff>