TheorySpace

Research

What I'm working on

This site is organised by method, since the same handful of methods keep turning up whatever the industry. These are the four I keep coming back to.

There's one question underneath all of them. When several models fit the data about equally well, which one do you trust? My usual way in is to simulate from a process I know, check against data the model hasn't seen, and write the assumptions down where people can argue with them.

Method areas and open questions

Bayesian models of choice and preference

Hierarchical Bayes for conjoint and MaxDiff, and the demand simulators built on top of them. This is my day job, so it's where I have the most opinions.

  • With a hundred respondents, how much work should the prior be allowed to do?
  • How do you carry posterior uncertainty through to a share forecast that people can still read?

Analytic agents and their evaluation

Agents that pull evidence together, suggest hypotheses and run the analysis. What I mostly want to know is how you'd check their work.

  • What would a fair test of an analysis agent look like?
  • When does a team of agents beat one careful prompt, and is it worth what it costs?

Causal inference

Effects from observational data, with the assumptions written down and a look at how hard the answer leans on each one.

  • How much unmeasured confounding would it take to flip the result?
  • When is it worth modelling the mechanism, and when is adjusting for covariates enough?

Adaptive experiments

Bandit and sequential designs next to plain balanced ones. Adaptive designs save sample, and I'd like to know what that costs you when it's time to estimate something.

  • How much sample do you really save at a fixed precision?
  • Which estimators still behave after adaptive allocation?

Ground rules

Data
Public datasets, or simulations where I state the generating process. No client data, ever.
Code
Every result links to a repo that reproduces it.
Baselines
Each method gets compared with the simplest thing that could plausibly work.
Limits
If something failed or came out null, that goes in the write-up too.

Collaboration

If you have a dataset or a domain problem where one of these methods might help, I'd love to hear about it. I'm at jay@theoryspace.ai.