Work / personal

Model Builder

Chat-driven ML training where an LLM picks the methodology and hand-written sklearn/LightGBM code does the training.

Kind
personal
Status
ongoing
Role
contributor, ongoing collaboration
Period
Jul 2026 – present
Affiliation
with Joe DiPietro (WPI)
domains
tabular automl llm-systems
methods
tool-use-orchestration methodology-registry gradient-boosting ensembling
stack
python fastapi uvicorn sqlite sqlalchemy pandas scikit-learn lightgbm xgboost prophet nextjs react typescript tailwind anthropic-api

Problem

Plenty of engineers can write code and stand up infrastructure but stall on the ML-specific judgment: how to frame the problem, which methodology suits the data, what “good” even looks like for this target. Existing AutoML tools mostly assume that framing is already done. You arrive knowing the task type and the target column. The tool searches hyperparameters from there.

Model Builder targets the step before that. You upload a CSV and describe the goal in plain language. The system does the triage a senior ML engineer would do: interpret the problem, pick an approach and explain why.

Data

  • Input: user-supplied tabular CSV; a profiling pass characterises columns before any modelling decision is made
  • v1 scope: binary/multiclass classification and regression on tabular data
  • Demo fixtures: samples/churn.csv and samples/house_prices.csv, generated by a script in the repo rather than committed as opaque blobs
  • Storage: SQLite, local files, single user, no auth in v1

Approach

The central design decision is a split of authority: the LLM decides, deterministic code executes. A Claude orchestrator using tool-use reads the data profile and the conversation, then selects and parameterises an entry from a curated methodology registry. It never generates training code.

That registry is the guardrail. Rather than an unbounded search space, methodologies are declared as YAML specs, twelve of them at the time of writing:

  • Classification: logistic baseline, random forest, LightGBM, XGBoost
  • Regression: linear baseline, random forest, LightGBM, XGBoost
  • Forecasting: LightGBM, Prophet
  • Ensembles: blending, stacking

Everything downstream is hand-written scikit-learn/LightGBM. The payoff is that behaviour stays predictable, debuggable and explainable. An LLM that emits training code is very hard to trust; an LLM that picks a known-good recipe and justifies the choice is much easier to audit.

The backend is FastAPI with SSE streaming so reasoning arrives incrementally rather than as a wall of text after a delay. The frontend is a Next.js chat UI that renders structured cards for the data profile, the proposed plan and the results, instead of a bare chat transcript.

Every completed run emits a bundle: the fitted model, a standalone train.py that reproduces the training end to end and a FastAPI inference stub. The train.py is the “glass box” commitment made concrete: the output is inspectable code you own, not an opaque endpoint.

Later work in the repo adds a trust layer that evaluates results with more scrutiny before recommending them, tournaments that pit methodologies against each other to build ensembles and in-app deployment behind an API.

Experiments

What didn’t work

Artifacts