Remove notion of "custom catalogs" from agent SDK

[nan-yu]

Description

The basic catalog maintained by the A2UI team has no difference from third-party catalogs.

This PR removes the notion of custom catalogs. Each catalog provided at runtime should be independent and immutable. At build time, catalogs can refer to components from other catalogs. They need to be bundled into a
free-standing one using the tools/build_catalog/build_catalog.py script.

Fixes: #650

Pre-launch Checklist

• I signed the CLA.
• I read the Contributors Guide.
• I read the Style Guide.
• I have added updates to the CHANGELOG.
• I updated/added relevant documentation.
• My code changes (if any) have tests.

If you need help, consider asking for advice on the discussion board.

[gemini-code-assist]

Code Review

This is a substantial but well-executed refactoring that removes the concept of "custom catalogs" needing runtime resolution, in favor of pre-bundled, self-contained catalogs. The changes centralize schema and validation logic, simplify the A2uiSchemaManager API, and improve the overall architecture. The test suite has been commendably updated to cover the new structure. I have one suggestion for a minor performance optimization.

[gspencergoog]

/gemini summary

[gemini-code-assist]

Summary of Changes

This pull request significantly refactors the A2UI SDK's approach to managing UI component catalogs. By eliminating the distinction of 'custom catalogs' and introducing a unified A2uiCatalog and CatalogConfig, the system now treats all catalogs as independent and immutable at runtime, with build-time resolution handled separately. The changes also modularize validation and payload fixing into dedicated classes, enhancing the robustness and clarity of the A2UI framework. These updates streamline agent configuration and ensure consistent handling of UI schemas and examples across the SDK.

Highlights

• Unified Catalog Management: The concept of 'custom catalogs' has been removed, and all catalogs are now managed uniformly through a new A2uiCatalog object and CatalogConfig. This simplifies how agents handle different sets of UI components.
• Modular Validation and Fixing: Schema validation and JSON payload fixing logic have been refactored into dedicated classes: A2uiValidator and A2uiPayloadFixer. This improves code organization, reusability, and maintainability.
• Streamlined Toolset Integration: The SendA2uiToClientToolset has been updated to directly utilize the new A2uiCatalog and A2uiPayloadFixer, replacing the previous a2ui_schema parameter with a2ui_catalog and introducing a2ui_examples for better prompt generation.
• Enhanced Schema Validation: The A2uiValidator now performs comprehensive checks including component integrity (unique IDs, root existence, valid references), topology (no circular references, no orphaned components), and recursion limits for both global depth and function calls.
• Absolute Data Model Paths: All example JSON files across various samples have been updated to use absolute paths (e.g., /imageUrl instead of imageUrl) for data model references, ensuring consistency and clarity.
• Simplified Agent Configuration: Agent configurations, particularly in samples like rizzcharts, now leverage the A2uiSchemaManager to load and manage catalogs and examples, reducing boilerplate and centralizing schema-related logic.

Changelog

• a2a_agents/python/a2ui_agent/src/a2ui/extension/a2ui_extension.py
  • Updated the parameter name accepts_inline_custom_catalogs to accepts_inline_catalogs in get_a2ui_agent_extension.
• a2a_agents/python/a2ui_agent/src/a2ui/extension/send_a2ui_to_client_toolset.py
  • Updated SendA2uiToClientToolset constructor and examples to use a2ui_catalog and a2ui_examples instead of a2ui_schema.
  • Removed imports for a2ui_schema_utils and validation.
  • Added import for A2uiCatalog.
  • Modified type aliases from A2uiSchemaProvider to A2uiCatalogProvider and introduced A2uiExamplesProvider.
  • Refactored _SendA2uiJsonToClientTool to manage a2ui_catalog and a2ui_examples.
  • Replaced direct schema instruction appending with a2ui_catalog.render_as_llm_instructions() and examples.
  • Replaced manual JSON parsing and fixing with a2ui_catalog.payload_fixer.fix().
• a2a_agents/python/a2ui_agent/src/a2ui/extension/validation.py
  • Removed the file as its validation logic was moved to a2ui/inference/schema/validator.py.
• a2a_agents/python/a2ui_agent/src/a2ui/inference/schema/catalog.py
  • Renamed CustomCatalogConfig to CatalogConfig and updated its documentation.
  • Added a for_basic_catalog class method to CatalogConfig.
  • Added a payload_fixer property to A2uiCatalog.
  • Removed the static method resolve_schema.
• a2a_agents/python/a2ui_agent/src/a2ui/inference/schema/constants.py
  • Added constants VERSION_0_8 and VERSION_0_9 for schema version strings.
• a2a_agents/python/a2ui_agent/src/a2ui/inference/schema/manager.py
  • Updated imports to use CatalogConfig.
  • Refactored A2uiSchemaManager initialization to accept a list of CatalogConfig objects.
  • Modified _supported_catalogs to be a list of A2uiCatalog instances.
  • Simplified basic catalog loading and removed custom catalog processing logic.
  • Updated _determine_catalog logic to reflect the new catalog management approach and error handling.
  • Modified get_agent_extension to iterate over the list of supported catalogs.
• a2a_agents/python/a2ui_agent/src/a2ui/inference/schema/payload_fixer.py
  • Added new file payload_fixer.py to encapsulate JSON payload fixing logic, including _remove_trailing_commas and fix methods.
• a2a_agents/python/a2ui_agent/src/a2ui/inference/schema/validator.py
  • Moved JSON Pointer pattern, recursion limits, and component constants to this file.
  • Enhanced A2uiValidator with detailed documentation on validation checks.
  • Updated _build_0_8_validator and _build_0_9_validator to use the referencing library for schema resolution.
  • Implemented a validate method to perform schema validation, component integrity, topology, and recursion checks.
  • Added helper functions: _find_root_id, _validate_component_integrity, _validate_topology, _extract_component_ref_fields, _get_component_references, _get_refs_recursively, and _validate_recursion_and_paths.
• a2a_agents/python/a2ui_agent/tests/extension/test_send_a2ui_to_client_toolset.py
  • Removed TEST_A2UI_SCHEMA constant.
  • Updated test_toolset_init_bool, test_toolset_init_callable, test_toolset_init_async_callable to use A2uiCatalog mocks and a2ui_examples.
  • Modified test_toolset_get_tools_enabled and test_toolset_get_tools_disabled to pass A2uiCatalog mocks.
  • Updated test_send_tool_init and test_send_tool_get_declaration to use a2ui_catalog and a2ui_examples.
  • Replaced test_send_tool_get_a2ui_schema with test_send_tool_resolve_catalog and test_send_tool_resolve_examples.
  • Modified test_send_tool_process_llm_request to assert on rendered_catalog and examples.
  • Updated test_send_tool_run_async_valid, test_send_tool_run_async_valid_list, test_send_tool_run_async_invalid_json, test_send_tool_run_async_schema_validation_fail to use payload_fixer.fix mock.
  • Removed test_send_tool_run_async_handles_trailing_comma.
• a2a_agents/python/a2ui_agent/tests/inference/test_modifiers.py
  • Updated test_manager_with_modifiers to correctly iterate through the _supported_catalogs list.
• a2a_agents/python/a2ui_agent/tests/inference/test_payload_fixer.py
  • Added new test file test_payload_fixer.py to verify the functionality of A2uiPayloadFixer.
• a2a_agents/python/a2ui_agent/tests/inference/test_resolve_catalog_schema.py
  • Removed the file as A2uiCatalog.resolve_schema was removed.
• a2a_agents/python/a2ui_agent/tests/inference/test_schema_manager.py
  • Updated imports to use CatalogConfig.
  • Modified test_manager_init_with_basic_catalog and test_manager_init_with_local_catalog to reflect _supported_catalogs as a list.
  • Updated test_manager_init_with_custom_catalog to use CatalogConfig and assert on list elements.
  • Modified test_determine_catalog_logic to reflect the new catalog determination rules and error messages.
• a2a_agents/python/a2ui_agent/tests/inference/test_validator.py
  • Updated catalog_0_9 fixture to include UpdateDataModelMessage and expanded component definitions.
  • Updated catalog_0_8 fixture with more detailed component definitions and dataModelUpdate.
  • Added test_catalog fixture for parameterized testing across versions.
  • Added make_payload helper method for creating version-specific payloads.
  • Implemented new tests for various validation scenarios: duplicate IDs, missing root, dangling references, self-references, circular references, function call recursion depth, orphaned components, global recursion depth, template reachability, and invalid JSON Pointer paths.
  • Added a test for v0.8 custom root reachability.
• samples/agent/adk/contact_lookup/agent.py
  • Updated A2uiSchemaManager instantiation to use CatalogConfig.for_basic_catalog.
• samples/agent/adk/contact_lookup/examples/action_confirmation.json
  • Updated data model paths to be absolute (e.g., /actionTitle).
• samples/agent/adk/contact_lookup/examples/contact_card.json
  • Updated data model paths to be absolute (e.g., /imageUrl).
• samples/agent/adk/contact_lookup/examples/contact_list.json
  • Updated data model paths to be absolute (e.g., /imageUrl).
• samples/agent/adk/contact_lookup/prompt_builder.py
  • Updated A2uiSchemaManager instantiation to use CatalogConfig.for_basic_catalog.
• samples/agent/adk/contact_multiple_surfaces/agent.py
  • Updated A2uiSchemaManager instantiation to use CatalogConfig.for_basic_catalog.
• samples/agent/adk/contact_multiple_surfaces/examples/action_confirmation.json
  • Updated data model paths to be absolute (e.g., /actionTitle).
• samples/agent/adk/contact_multiple_surfaces/examples/chart_node_click.json
  • Updated data model paths to be absolute (e.g., /imageUrl).
• samples/agent/adk/contact_multiple_surfaces/examples/contact_card.json
  • Updated data model paths to be absolute (e.g., /imageUrl).
• samples/agent/adk/contact_multiple_surfaces/examples/contact_list.json
  • Updated data model paths to be absolute (e.g., /imageUrl).
• samples/agent/adk/contact_multiple_surfaces/examples/multi_surface.json
  • Updated data model paths to be absolute (e.g., /imageUrl).
• samples/agent/adk/contact_multiple_surfaces/examples/org_chart.json
  • Updated data model paths to be absolute (e.g., /hierarchy).
• samples/agent/adk/contact_multiple_surfaces/prompt_builder.py
  • Updated A2uiSchemaManager instantiation to use CatalogConfig.for_basic_catalog.
• samples/agent/adk/restaurant_finder/agent.py
  • Updated A2uiSchemaManager instantiation to use CatalogConfig.for_basic_catalog.
• samples/agent/adk/restaurant_finder/examples/booking_form.json
  • Updated data model paths to be absolute (e.g., /title).
• samples/agent/adk/restaurant_finder/examples/confirmation.json
  • Updated data model paths to be absolute (e.g., /title).
• samples/agent/adk/restaurant_finder/examples/single_column_list.json
  • Updated data model paths to be absolute (e.g., /title).
• samples/agent/adk/restaurant_finder/examples/two_column_list.json
  • Updated data model paths to be absolute (e.g., /title).
• samples/agent/adk/restaurant_finder/prompt_builder.py
  • Updated A2uiSchemaManager instantiation to use CatalogConfig.for_basic_catalog.
• samples/agent/adk/rizzcharts/main.py
  • Updated imports for A2uiSchemaManager and CatalogConfig.
  • Modified main function to instantiate A2uiSchemaManager with a list of CatalogConfig objects.
  • Updated RizzchartsAgent initialization to pass base_url, schema_manager, a2ui_catalog_provider, and a2ui_examples_provider.
  • Refactored RizzchartsAgentExecutor initialization to use schema_manager.
  • Updated A2AStarletteApplication to retrieve the agent card from the RizzchartsAgent instance.
• samples/agent/adk/rizzcharts/agent.py
  • Updated imports for AgentCapabilities, AgentCard, AgentSkill, A2uiCatalogProvider, A2uiExamplesProvider, and A2uiSchemaManager.
  • Removed wrap_as_json_array import.
  • Defined ROLE_DESCRIPTION, WORKFLOW_DESCRIPTION, and UI_DESCRIPTION as constants.
  • Modified RizzchartsAgent constructor to accept base_url, schema_manager, a2ui_catalog_provider, and a2ui_examples_provider.
  • Updated RizzchartsAgent to use schema_manager.generate_system_prompt for instructions.
  • Modified SendA2uiToClientToolset instantiation to use a2ui_catalog and a2ui_examples.
  • Removed get_a2ui_schema and load_example methods.
  • Updated get_agent_card to use schema_manager.get_agent_extension().
• samples/agent/adk/rizzcharts/agent_executor.py
  • Updated imports to reflect new schema management.
  • Renamed internal state keys for A2UI catalog and added one for examples.
  • Renamed get_a2ui_schema to get_a2ui_catalog and added get_a2ui_examples.
  • Refactored RizzchartsAgentExecutor constructor to use A2uiSchemaManager.
  • Modified _prepare_session to retrieve effective catalog and examples from schema_manager.
  • Updated session state updates to store the A2uiCatalog object and examples string.
• samples/agent/adk/rizzcharts/component_catalog_builder.py
  • Removed the file as its functionality is now handled by A2uiSchemaManager.
• samples/agent/adk/rizzcharts/examples/rizzcharts_catalog/chart.json
  • Updated data model paths to be absolute (e.g., /chart.title).
• samples/agent/adk/rizzcharts/examples/rizzcharts_catalog/map.json
  • Updated data model paths to be absolute (e.g., /mapConfig.center).
• samples/agent/adk/rizzcharts/examples/standard_catalog/chart.json
  • Updated data model paths to be absolute (e.g., /chart.title).
• samples/agent/adk/rizzcharts/examples/standard_catalog/map.json
  • Updated data model paths to be absolute (e.g., /mapConfig.locations).
• samples/agent/adk/rizzcharts/rizzcharts_catalog_definition.json
  • Inlined standard catalog component definitions and added catalogId to the root of the JSON.
• specification/v0_8/eval/GEMINI.md
  • Updated documentation to refer to component object instead of componentProperties.
• tools/build_catalog/README.md
  • Clarified in the README that only common_types are automatically resolved by the A2UI SDK.

[jacobsimionato]

Can we move this to a top-level function in a different file, perhaps even a different folder, e.g. inference/basic_catalog or something?

That makes the conceptual separation clear, and avoids us accidentally sprinkling basic catalog assumptions into the core library. Ideally, it would be possible for us to publish basic catalog support as a completely separate library which depends on the core inference library.

[wrenj]

+1 on this, then we can run the basic catalog through the sdk like any other catalog

[nan-yu]

This part is refactored with 2 class methods in the CatalogConfig:

• CatalogConfig.bundled(version, examples_path): load the bundled (basic) catalog with examples.
• CatalogConfig.from_path(name, catalog_path, examples_path): load the user catalog with examples.

I this way, the basic catalog can be loaded similar as other catalogs.

[jacobsimionato]

Thanks for iterating!!

I'd still much prefer moving all references to the basic catalog to a separate basic_catalog folder and inverting the dependency structure, so that the basic catalog support depends on the core SDK. I can see how what you have is a super ergonomic way to access the basic catalog, but that's almost part of the problem, because it privileges the basic catalog above other catalogs. We have had a similar issue in the client SDKs and it's been very difficult to unwind, so that's why I'm so sensitive to it!

E.g.

BasicCatalog.getConfig(version, example_paths)

or

CatalogConfig(BasicCatalog.provider(version), example_paths)

I don't feel strongly about the exact API, as long as the basic catalog stuff is in a separate folder which depends on the core SDK, rather than the other way around.

Rationale:

• Avoid giving developers the idea that they should use the basic catalog if they are using A2UI
• Avoid us accidentally baking in more basic catalog assumptions. It's easy to spot when depend on it if it's in a different folder.
• Allow us to release basic catalog as a separate package if we want
• Provide an example for other catalog developers for how to expose their catalog programmatically in the Python SDK. E.g. if there is a Material Catalog, they might want to implement a package to expose it ergonomically in Python. If you provide a separate basic catalog implementation with programmatic APIs that build on top of the core APIs, they can trivially follow your example.

[nan-yu]

Thanks for the explanation. That makes a lot of sense.

I've refactored the SDK to decouple basic catalog support from the core library:

• Moved the basic_catalog loading and instantiation to a separate basic_catalog folder.
• Inverted the dependency to make basic_catalog depend on the core SDK.
• Updated pack_specs_hook.py to generalized module loading. Basic catalog is currently bundled into the SDK the same way as other schemas (server-to-client and common-types), but it is modularized to be easily replaced with other approach if the basic catalog is moved outside of the specification directory.

[nan-yu]

All comments are addressed. Ready for another round of review. Thanks @jacobsimionato @wrenj