MCP App Store
by KeenEthics
NeoSigma icon

NeoSigma MCP Server for ChatGPT

By NeoSigma

Overview

NeoSigma gives your coding agent access to your AI agent's production traces, issues, datasets, evals and experiments, so it can find what is going wrong and test a fix next to your code. Use it to: - Search production sessions and read traces to find failures, including runs that look successful but are not. - Read the issues NeoSigma found and the sessions behind each one. - Turn failing sessions into proposed evals, review them, and save them to a dataset. - Run experiments on a dataset and compare results across agent versions. - Add NeoSigma tracing to your agent, import your existing checks as verifiers, and set up the environment your evals run in. Who it is for: teams that build and run AI agents and already send traces to NeoSigma, or want to start. Limits: you need a NeoSigma account, and you sign in on first use. The plugin reads and changes data only in the NeoSigma projects your account can access. It does not change your agent in production. Running an experiment uses your organization's NeoSigma usage.

How to connect

  1. ChatGPT: Open the NeoSigma listing in the ChatGPT app directory and choose Connect. ChatGPT listing NeoSigma exposes 53 tools here.

Tools

create_dataset

ChatGPT
Create a project-owned dataset before publishing Harbor tasks to it.

create_eval_from_proposal

ChatGPT
Keep one proposed eval in a dataset, creating it unless it already exists. Call it only for proposals the user approved. created is false when the proposal matched an eval the project already has, and that eval can replay a different request than the proposal's instruction. Creating the same proposal again returns the same eval, so a retry is safe.

get_automation

ChatGPT
Read one automation in full, with its settings and whether it is on.

get_chat

ChatGPT
Read one agent conversation, with every message and the state of its runs. Read it again until the most recent run completes to get the agent's reply.

get_dataset

ChatGPT
Read one dataset: its name, its description and where it came from.

get_document

ChatGPT
Read one knowledge document: its title and its Markdown body.

get_eval_run

ChatGPT
Deprecated. Use get_experiment to read experiment progress and results.

get_experiment

ChatGPT
Read one experiment: its state, its dataset, its trial counts and its cost. Read list_experiment_trials for why each trial passed or failed.

get_experiment_trial

ChatGPT
Read one trial in full, including the instruction the agent received.

get_issue

ChatGPT
Read one issue in full, with its description and its current state.

get_project_setup

ChatGPT
Read a project's setup script, the version its runs use, and its verification state. Start here. content is the file at path at the default branch head of the workspace's primary repository. Edit that file, push a branch, and verify it with verify_project_setup and primary_ref. Runs use a version only after it passes on the default branch. While run_usable is null, the project's runs use the workspace setup.

get_project_setup_verification

ChatGPT
Read one verification: its lifecycle, its verdict, and repair diagnostics. A failed outcome is a script fault to repair. A failed lifecycle is a sandbox fault that a script edit cannot fix.

get_session

ChatGPT
Read one session: its status, its summary, its turn count, its duration and its cost.

get_session_observation

ChatGPT
Read the diagnosis NeoSigma recorded for a session, with the traces that support it.

get_setup_verification

ChatGPT
Deprecated. Use get_project_setup_verification for a project verification. Reads one verification: its lifecycle, its verdict, and repair diagnostics. A failed outcome is a script fault to repair. A failed lifecycle is a sandbox fault that a script edit cannot fix.

get_turn

ChatGPT
Read one turn: its messages, its model, its cost, its tokens and its link. Share the returned url with the user so they can open the turn in NeoSigma.

get_workspace_setup

ChatGPT
Deprecated. Use get_project_setup. Each project gets its own setup script. Reads a workspace's setup script, its verification state, and its env var names. When source is "repository", content is .neosigma/workspaces/<workspace_id>/setup.sh at the default branch head. Edit that file, push a branch, and verify it with primary_ref. When source is "stored", save against the returned head.id and head.object_version. Environment variable values are never returned.

list_automation_runs

ChatGPT
List automation runs, newest first, with each run's outcome and error.

list_automations

ChatGPT
List the organization's automations, each with its newest run. A scheduled eval run is an automation whose trigger type is schedule. Its definition.trigger holds the cron expression and time zone, and enabled is false while it is paused.

list_dataset_tasks

ChatGPT
List the tasks a dataset holds, which are the cases an experiment measures. Read them to know what an experiment will exercise before it starts.

list_datasets

ChatGPT
List a project's evaluation datasets, with the identifier of each one. A dataset is the set of cases an experiment measures, so its identifier is what run_experiment takes.

list_documents

ChatGPT
List a workspace's knowledge documents, most recently changed first. These are the runbooks, glossaries and conventions the team wrote about their project. Each item has no body, so read the relevant ones with get_document before answering a project-specific question.

list_eval_runs

ChatGPT
Deprecated. Use list_experiments to list a project's experiments. This tool still lists legacy evaluation runs, newest first.

list_experiment_trials

ChatGPT
List an experiment's trials: one attempt of one task, with its verdict and why. Each trial has its verdict, its verifier rewards, the judge's summary and the reason when it produced no result. Read get_experiment_trial for the agent's output and the judge's full reasoning.

list_experiments

ChatGPT
List a project's experiments, newest first.

list_harnesses

ChatGPT
List the harnesses an experiment may use. Each item carries the identifier run_experiment takes in harness.

list_issues

ChatGPT
List the organization's issues, the failure modes recorded against it. Each issue carries its display id, its title, its severity and its state.

list_models

ChatGPT
List models available to agent-backed experiments. The list omits models reserved for verifier evaluations.

list_projects

ChatGPT
List a workspace's projects, with the identifier of each one. A project owns its datasets and experiments, so its identifier is what list_datasets, list_experiments and run_experiment take.

list_session_turns

ChatGPT
List a session's turns, oldest first. A message cut to the list limit is flagged as truncated. Read the whole turn with get_turn, and its spans with list_trace_spans and the turn's trace identifier.

list_trace_spans

ChatGPT
Read one chronological page of a trace's spans. Each span carries its timing, its status and its attributes. Read the spans to find what an agent session did, one step at a time.

list_vaults

ChatGPT
List a workspace's credential vaults, with the identifier of each one. Each item carries the identifier run_experiment takes in vault_ids.

list_verifiers

ChatGPT
List one project's verifiers, with identifier, state and source.

list_workspace_env_vars

ChatGPT
List a workspace's own environment variables, by name. Values are never returned. A setup boot of the workspace or one of its projects receives these and the shared variables from list_workspace_env_var_links.

list_workspaces

ChatGPT
List the organization's workspaces, with the identifier of each one. Start here to find the workspace_id that list_projects takes.

propose_evals_from_sessions

ChatGPT
Propose evals from a project's failing sessions in a time window, and create none. Proposes one eval per distinct problem: failures that are the same problem share a proposal, a failure an existing eval already tests proposes that eval, and the rest are held back with a reason. Show the user each proposal's instruction and failure_text, and pass only the proposals they approve to create_eval_from_proposal. A project runs one proposal at a time, so wait for each call before the next. A large window can take minutes, so prefer a window of a day or less.

propose_evals_from_traces

ChatGPT
Deprecated. Use propose_evals_from_sessions, which takes the same input.

publish_harbor_task

ChatGPT
Validate, store in GCS, and create one immutable Harbor dataset task. No file is stored when validation fails. Reuse the same idempotency key and exact manifest to make a retry return the already-published task.

push_verifiers

ChatGPT
Push one repository's whole verifier set to a project in one call. The verifiers then belong to that project. A verifier of the project from the same repository that is not in the files is turned off, so never split one set across calls. The answer lists the verifiers the push created, updated, left unchanged and turned off.

reschedule_automation

ChatGPT
Change when a schedule automation runs. Its evals, channel and mentions stay. This changes the schedule for the whole team, so first tell the person the new times in plain words and call this only after they agree.

run_experiment

ChatGPT
Plan or start an experiment that runs a dataset's tasks on an agent. A run starts real agent sessions and costs money. Call it with dry_run true first, show the user the plan, and start it with dry_run false only after they confirm. The run is dispatched, not completed. Read get_experiment for its progress.

save_workspace_setup

ChatGPT
Deprecated. Keep the script at .neosigma/workspaces/<workspace_id>/setup.sh. Commit that file to the primary repository and call verify_workspace_setup. This still saves a new setup-script version as the editable head. Workloads keep an earlier verified or person-saved version until this one passes verify_workspace_setup. Without one, workloads may select this version, but an eval does not start until a version passes. A stale head is refused. Call get_workspace_setup again and reapply the change.

search_sessions

ChatGPT
Search a project's recorded agent sessions with a plain-language question. Returns the best matches first, each with its relevance, its summary and the snippets that matched. Open a match with get_session and list_session_turns.

set_automation_enabled

ChatGPT
Pause or resume an automation. A paused schedule does not run. This affects the whole team, so confirm with the person before calling.

setup_benchmark_conversion

ChatGPT
Prepare a public benchmark with the convert-benchmark-to-harbor skill. Conversion is not a managed NeoSigma worker. The caller's agent installs the public skill and uses the two CLI commands it names.

setup_telemetry

ChatGPT
Set up NeoSigma telemetry in the caller's application. Returns an instruction to install and use the integrate-sdk skill. Follow it to complete the integration in the caller's own codebase.

setup_tracing

ChatGPT
Set up NeoSigma tracing in the caller's application. Returns an instruction to install and use the integrate-sdk skill. Follow it to complete the integration in the caller's own codebase.

setup_verifiers

ChatGPT
Import a codebase's existing checks into NeoSigma as verifiers. Returns an instruction to install and use the import-verifiers skill, which writes the repository's verifier YAML set and applies it with push_verifiers.

suggest_verifier

ChatGPT
Suggest a change to a verifier, or a new verifier, for a person to review. Nothing changes until a person accepts it. A project keeps one open suggestion per slug, so suggesting the same slug again replaces it, and the answer reports created false.

validate_harbor_task

ChatGPT
Validate a complete Harbor task without persisting any files or rows. Use this after changing a local conversion and before calling publish_harbor_task. The response is derived from Harbor's parser.

verify_project_setup

ChatGPT
Boot the project's setup script in a sandbox against exact pushed commits. Verifies the setup script at path at primary_ref or the default branch. Local, unpushed changes are not verified. Poll get_project_setup_verification until the lifecycle is completed, failed, or cancelled.

verify_workspace_setup

ChatGPT
Deprecated. Use verify_project_setup to verify a project's own setup script. Boots the setup script in a sandbox against exact pushed commits. Verifies .neosigma/workspaces/<workspace_id>/setup.sh at primary_ref or the default branch. Without that file, boots the stored version setup_script_id. Local, unpushed changes are not verified. Poll get_setup_verification until the lifecycle is completed, failed, or cancelled.

Capabilities

Writes

Example Prompts

Click any prompt to copy it.

Similar Apps

Listed on MCP App Store

Listed on MCP App Store

Add this badge to your site or README. It links to this page.

HTML

<a href="https://mcp-app-store.keenethics-labs.com/app/neosigma"><img src="https://mcp-app-store.keenethics-labs.com/badges/listed-on-mcp-app-store.svg" alt="Listed on MCP App Store" width="220" height="44"></a>

Markdown

[![Listed on MCP App Store](https://mcp-app-store.keenethics-labs.com/badges/listed-on-mcp-app-store.svg)](https://mcp-app-store.keenethics-labs.com/app/neosigma)