AgentBlock/docs/visa-backend-database-design.md

# VISA Backend Database Design

## Purpose

The project should treat VISA as a backend model specification protocol, not as a set of frontend tables. The database is the source of truth for reproducible ABM descriptions. The frontend consumes derived view models for graph editing, summaries, validation reports, and export actions.

This design follows the selected approach C:

```text
Database and backend: VISA-native T1-T8 specification
Adapter API: VISA data -> frontend workflow view model
Frontend: modeling interface, not raw database table display
```

## High-Level Architecture

```text
Frontend
  Canvas, forms, summaries, validation report, export actions
        |
        v
Adapter API
  Builds frontend view models from VISA records
  Converts semantic edit actions into VISA updates
        |
        v
Backend Services
  Model versioning
  VISA T1-T8 CRUD
  19-rule consistency checking
  VISA JSON export
  Future code generation
        |
        v
Database
  Model/version metadata
  VISA semantic modules
  Normalized helper tables
  Consistency check results
```

## System Tables

### `models`

Stores the model project.

```text
id
name
description
domain
author
created_at
updated_at
```

### `model_versions`

Stores versioned VISA specifications. Every VISA record belongs to one `model_version_id`.

```text
id
model_id
version_name
status: draft | checked | released | archived
source_protocol: VISA
created_by
created_at
notes
```

## VISA Semantic Modules

The database may use more than eight physical tables, but the backend must expose eight VISA semantic modules.

## T1 Agent

Physical table: `visa_agents`

```text
id
model_version_id
name
symbol_set
instance_symbol
category: Environment | Space | Decision-maker | Passive
description
quantity_symbol
quantity_type: fixed | variable
quantity_value
sort_order
```

Design rule: database records should represent model-semantic agents, not implementation classes. Datasets, output records, and reports belong to T5 or T6, not T1.

## T2 Variable

Physical table: `visa_variables`

```text
id
model_version_id
agent_id
name
symbol
variable_type: exog_homo | exog_hetero | endog_decision | endog_non_decision
data_type
value_source
unit
description
is_time_indexed
sort_order
```

Design rule: endogenous variables should be traceable to T4 function IDs. Exogenous variables marked as `Input` must be covered by T6a.

## T3 Sensing

Physical tables:

```text
visa_sensing_relations
visa_sensing_variables
```

`visa_sensing_relations`

```text
id
model_version_id
observer_agent_id
observed_agent_id
access_type: none | partial | all
scope: self | peer | other | all | singleton
condition
notes
```

`visa_sensing_variables`

```text
sensing_relation_id
variable_id
```

Design rule: store sensing as normalized relations rather than a wide matrix. The backend can reconstruct the VISA matrix for export and can check whether T4 functions use only authorized information.

## T4 Internal Function

Physical tables:

```text
visa_internal_functions
visa_function_inputs
visa_function_updates
```

`visa_internal_functions`

```text
id
model_version_id
agent_id
function_code
name
method
reference
description
sort_order
```

`visa_function_inputs`

```text
function_id
variable_id
source_type: self | sensed | input | output
expression
```

`visa_function_updates`

```text
function_id
target_agent_id
variable_id
update_type: self_state | external_effect | create_agent | remove_agent
expression
```

Design rule: every internal function must update at least one endogenous variable or create/remove an instance of a variable-quantity agent type.

## T5 Associated Data

Physical table: `visa_associated_data`

```text
id
model_version_id
data_code
title
data_type: Empirical | Literature | Generated
temporal_type: Static | Dynamic
source
collection_method: Survey | Administrative | Sensor | Experimental | Computational
preprocessing_method: None | Selected | Aggregated | Transformed
record_count
availability: Open | Restricted | Private
license
url_or_reference
description
```

Design rule: this table records data provenance. It is not merely a source-column lookup table.

## T6 Input and Output

Physical tables:

```text
visa_inputs
visa_outputs
```

`visa_inputs`

```text
id
model_version_id
agent_id
variable_id
symbol
value_or_distribution
data_source_id
derivation: Direct | Estimated | Computed | Assumed
algorithm
reference
notes
```

`visa_outputs`

```text
id
model_version_id
symbol
indicator_name
formula
data_type
unit
frequency
description
```

Design rule: T6a covers exogenous model inputs. T6b covers simulation output indicators. Component-to-component frontend wiring is a derived view, not the canonical T6 design.

## T7 Schedule

Physical tables:

```text
visa_schedule_steps
visa_termination_conditions
```

`visa_schedule_steps`

```text
id
model_version_id
step_number
agent_id
function_id
execution_mode: Synchronous | Sequential | Random-order | Asynchronous
execution_parameters
condition
sort_order
```

`visa_termination_conditions`

```text
id
model_version_id
condition_code
indicator_symbol
condition_expression
description
value_source
termination_logic
```

Design rule: execution mode must be structured because synchronization choices can change ABM behavior.

## T8 Validation

Physical table: `visa_validations`

```text
id
model_version_id
validation_code
level: Agent | Model | Output
validation_object
benchmark_data_id
method
indicator
passing_condition
reference
status: pending | passed | failed | blocked
notes
```

Design rule: T8 is scientific model validation. It should not be mixed with the 19-rule consistency checker.

## Consistency Check Results

Physical table: `visa_consistency_check_results`

```text
id
model_version_id
rule_code
rule_type: within_table | cross_table
status: passed | failed | warning
message
related_table
related_record_id
checked_at
```

The backend should run the VISA 19-rule checker against a complete model version and persist results here. These records drive frontend check reports.

## Frontend Boundary

The frontend must not present the database as eight raw table pages. It should provide modeling workflows:

```text
Model overview
Agent and space designer
Variable editor
Behavior/function designer
Data source manager
Schedule builder
Validation and check report
Export/generate actions
```

These screens are allowed to show summaries, forms, cards, graphs, and reports. They should not make the user edit `visa_agents` or `visa_variables` as database rows.

## Adapter Contract

The adapter layer should expose two families of API responses.

Canonical VISA:

```text
GET /api/model-versions/:id/visa
```

Returns a complete T1-T8 VISA JSON object for checking, export, and code generation.

Frontend view model:

```text
GET /api/model-versions/:id/view-model
```

Returns graph nodes, graph edges, grouped variables, behavior cards, schedule timeline items, and validation summaries derived from VISA records.

Semantic edit actions:

```text
POST /api/model-versions/:id/actions/add-agent
POST /api/model-versions/:id/actions/add-variable
POST /api/model-versions/:id/actions/add-function
POST /api/model-versions/:id/actions/run-check
```

The backend translates these actions into normalized VISA writes and refreshes the derived view model.

## First Implementation Step

The current frontend can remain on `WorkflowModel` while a VISA adapter is introduced. The adapter converts:

```text
T1 Agent -> workflow nodes
T3 Sensing + T4 Internal Function -> view edges and behavior sections
T5 Associated Data -> associated data rows
T6 Input/Output -> I/O rows
T7 Schedule -> schedule rows
T8 Validation -> validation rows
```

This keeps the existing UI functional while moving the source of truth toward the backend VISA model.