AI eats project planning

At the moment, you can’t swing a Schrödinger's cat without hitting some reference to AI. Such is the spooky action of ChatGPT and other LLMs.

With our new public forecaster, we’re actually employing a different breed of AI: Monte Carlo simulation. Monte Carlo is a statistical technique used to model complex systems across a wide range of fields—traffic control, stock predictions, and nuclear reactors, to name a few. The idea is to use the law of large numbers to predict how a complex system might behave.

Monte Carlo runs scenarios using our best empirical data and knowledge of the "rules of physics" for the system at hand. By collecting results from enough randomized scenarios, we can get an idea not only of what is most likely to occur, but also visualize the range of possible outcomes.

tl;dr: Monte Carlo is tailor-made for forecasting how long software projects will take to finish.

How Socratic’s forecaster works

The basic steps and inputs of our Monte Carlo model are as follows:

(1) Set up N tasks with random (but representative) durations.

(2) Set up M (identically-skilled) people to complete the tasks.

(3) Assign the N tasks to the M people.

(4) Determine the completion dates for each task.

(5) Construct a burndown chart from these dates.

(6) Repeat steps 1-5 Q times.

You provide the forecaster N and M; Socratic provides the rest. This includes randomly-sampled task durations based on our anonymized body of historical actuals, which is needed for step 1.

For our public forecaster, we’ve set the number of model runs (Q) to 100. The more you do, the more statistical significance you gain, but it also starts to absorb some serious compute cycles. In our own analysis, 100 runs is enough to get a reliable forecast.

A Monte Carlo model can be improved by expanding the list of effects that impact the system in question. You might start with a basic model, but iterate with finer and finer effects until you reach an arbitrary degree of accuracy. In the realm of project work and software engineering, these finer effects would be things like:

• The rate at which tasks fall idle;

• The number and relationship of blocked tasks;

• The amount of rework involved before a task is completed;

• The application of people with specialized skills.

Our public forecaster gives 'first order' approximations, a bit like ignoring air resistance in a physics problem. (Suffice to say, it’s wildly more advanced than any spreadsheet!)

The advanced forecaster built into the Socratic app incorporates effects like the ones listed above, coupled with 10x the number of model runs, to yield even richer forecasts.

So there it is. The next time someone asks how long some project might take to tackle, you can chuck the guesstimates and spreadsheets. With just a couple of simple inputs and a click, you get the power of Monte Carlo, backed by real-world data.

Make project estimating the machine’s problem.