NeoSigma MCP Server for ChatGPT
By NeoSigma
Overview
NeoSigma gives your coding agent access to your AI agent's production traces, issues, datasets, evals and experiments, so it can find what is going wrong and test a fix next to your code. Use it to: - Search production sessions and read traces to find failures, including runs that look successful but are not. - Read the issues NeoSigma found and the sessions behind each one. - Turn failing sessions into proposed evals, review them, and save them to a dataset. - Run experiments on a dataset and compare results across agent versions. - Add NeoSigma tracing to your agent, import your existing checks as verifiers, and set up the environment your evals run in. Who it is for: teams that build and run AI agents and already send traces to NeoSigma, or want to start. Limits: you need a NeoSigma account, and you sign in on first use. The plugin reads and changes data only in the NeoSigma projects your account can access. It does not change your agent in production. Running an experiment uses your organization's NeoSigma usage.
How to connect
- ChatGPT: Open the NeoSigma listing in the ChatGPT app directory and choose Connect. ChatGPT listing NeoSigma exposes 53 tools here.
Tools
create_eval_from_proposal
ChatGPTcreated is false when the proposal matched an eval the project already has, and that eval can replay a different request than the proposal's instruction. Creating the same proposal again returns the same eval, so a retry is safe.get_automation
ChatGPTget_chat
ChatGPTget_dataset
ChatGPTget_document
ChatGPTget_eval_run
ChatGPTget_experiment to read experiment progress and results.get_experiment
ChatGPTlist_experiment_trials for why each trial passed or failed.get_experiment_trial
ChatGPTget_issue
ChatGPTget_project_setup
ChatGPTcontent is the file at path at the default branch head of the workspace's primary repository. Edit that file, push a branch, and verify it with verify_project_setup and primary_ref. Runs use a version only after it passes on the default branch. While run_usable is null, the project's runs use the workspace setup.get_project_setup_verification
ChatGPTfailed outcome is a script fault to repair. A failed lifecycle is a sandbox fault that a script edit cannot fix.get_session
ChatGPTget_session_observation
ChatGPTget_setup_verification
ChatGPTget_project_setup_verification for a project verification. Reads one verification: its lifecycle, its verdict, and repair diagnostics. A failed outcome is a script fault to repair. A failed lifecycle is a sandbox fault that a script edit cannot fix.get_turn
ChatGPTurl with the user so they can open the turn in NeoSigma.get_workspace_setup
ChatGPTget_project_setup. Each project gets its own setup script. Reads a workspace's setup script, its verification state, and its env var names. When source is "repository", content is .neosigma/workspaces/<workspace_id>/setup.sh at the default branch head. Edit that file, push a branch, and verify it with primary_ref. When source is "stored", save against the returned head.id and head.object_version. Environment variable values are never returned.list_automation_runs
ChatGPTlist_automations
ChatGPTschedule. Its definition.trigger holds the cron expression and time zone, and enabled is false while it is paused.list_dataset_tasks
ChatGPTlist_datasets
ChatGPTrun_experiment takes.list_documents
ChatGPTget_document before answering a project-specific question.list_eval_runs
ChatGPTlist_experiments to list a project's experiments. This tool still lists legacy evaluation runs, newest first.list_experiment_trials
ChatGPTget_experiment_trial for the agent's output and the judge's full reasoning.list_experiments
ChatGPTlist_harnesses
ChatGPTrun_experiment takes in harness.list_issues
ChatGPTlist_models
ChatGPTlist_projects
ChatGPTlist_datasets, list_experiments and run_experiment take.list_session_turns
ChatGPTget_turn, and its spans with list_trace_spans and the turn's trace identifier.list_trace_spans
ChatGPTlist_vaults
ChatGPTrun_experiment takes in vault_ids.list_verifiers
ChatGPTlist_workspace_env_var_links
ChatGPTlist_workspace_env_vars. Values are never returned.list_workspace_env_vars
ChatGPTlist_workspace_env_var_links.list_workspaces
ChatGPTworkspace_id that list_projects takes.propose_evals_from_sessions
ChatGPTinstruction and failure_text, and pass only the proposals they approve to create_eval_from_proposal. A project runs one proposal at a time, so wait for each call before the next. A large window can take minutes, so prefer a window of a day or less.propose_evals_from_traces
ChatGPTpropose_evals_from_sessions, which takes the same input.publish_harbor_task
ChatGPTpush_verifiers
ChatGPTreschedule_automation
ChatGPTrun_experiment
ChatGPTdry_run true first, show the user the plan, and start it with dry_run false only after they confirm. The run is dispatched, not completed. Read get_experiment for its progress.save_workspace_setup
ChatGPT.neosigma/workspaces/<workspace_id>/setup.sh. Commit that file to the primary repository and call verify_workspace_setup. This still saves a new setup-script version as the editable head. Workloads keep an earlier verified or person-saved version until this one passes verify_workspace_setup. Without one, workloads may select this version, but an eval does not start until a version passes. A stale head is refused. Call get_workspace_setup again and reapply the change.search_sessions
ChatGPTget_session and list_session_turns.set_automation_enabled
ChatGPTsetup_benchmark_conversion
ChatGPTsetup_telemetry
ChatGPTsetup_tracing
ChatGPTsetup_verifiers
ChatGPTsuggest_verifier
ChatGPTcreated false.validate_harbor_task
ChatGPTpublish_harbor_task. The response is derived from Harbor's parser.verify_project_setup
ChatGPTpath at primary_ref or the default branch. Local, unpushed changes are not verified. Poll get_project_setup_verification until the lifecycle is completed, failed, or cancelled.verify_workspace_setup
ChatGPTverify_project_setup to verify a project's own setup script. Boots the setup script in a sandbox against exact pushed commits. Verifies .neosigma/workspaces/<workspace_id>/setup.sh at primary_ref or the default branch. Without that file, boots the stored version setup_script_id. Local, unpushed changes are not verified. Poll get_setup_verification until the lifecycle is completed, failed, or cancelled.Capabilities
Example Prompts
Click any prompt to copy it.
Links
Similar Apps
Listed on MCP App Store
Add this badge to your site or README. It links to this page.
HTML
<a href="https://mcp-app-store.keenethics-labs.com/app/neosigma"><img src="https://mcp-app-store.keenethics-labs.com/badges/listed-on-mcp-app-store.svg" alt="Listed on MCP App Store" width="220" height="44"></a>
Markdown
[](https://mcp-app-store.keenethics-labs.com/app/neosigma)






