Counting calories from a photo — how it works and how accurate it is

You photograph a plate and a few seconds later you have calories and macros. It looks like magic, but it is three describable steps — and three nameable sources of error. Here is what actually happens, what the number is worth, and when not to trust it.

Where the number comes from

Analysing a meal photo is not one operation but a chain of three. Each link can fail independently of the others and the errors compound, which is why it helps to know where each one lives.

1. Identifying the ingredients. The model works out what is on the plate: chicken breast, rice, broccoli, a spoonful of sauce. This is the part that works best — common, clearly visible foods are recognised reliably.
2. Estimating the portion. How many grams of that rice? Here the model works from proportions against the plate, the cutlery and everything else in frame. This is the weakest link in the whole chain.
3. Converting to nutrition. The recognised foods and estimated weights go to a nutrition database, which returns calories, protein, carbohydrate and fat.

Step three is essentially deterministic — 100 g of cooked white rice is about 130 kcal and that is that. All the uncertainty sits in steps one and two, and in practice mostly in step two.

Where the error actually lives

Volume is harder than naming

A photograph is flat and food is not. A mound of rice seen at 45 degrees could be 150 g or 250 g — from directly above it looks much the same. Models handle this by looking for reference points of known size in frame: the rim of the plate, a fork, a hand. The fewer of those there are, the wider the spread.

Fat is invisible

This is the single biggest problem. A tablespoon of olive oil in the pan is about 120 kcal, and in the finished dish you cannot see it at all. The same goes for butter in scrambled eggs, cream in soup and mayonnaise folded into a salad. A photo has no way to detect it, because there is nothing to see.

Hidden layers

A sandwich shows you the top slice, not what is under it. A bake shows you cheese, not the layer beneath. A stew shows you the top of the pot. The model guesses from what it can see, plus what statistically tends to accompany what it can see.

How big the error is in numbers

It is worth putting this next to the reference point people usually skip: a human estimating by eye is also wrong, and badly so. Doubly labelled water studies — which measure real energy expenditure independently of what anyone wrote down — have shown the same thing for decades: people under-report what they eat by 20 to 50 per cent on average. Not because they lie, but because they forget snacks and underestimate portions.

Typing entries into a conventional calorie counter does not remove the problem either. You pick “pasta, one serving” from a list and you get the value for a serving somebody else defined — not necessarily the one on your plate.

These are orders of magnitude rather than guarantees — they depend on what and how you eat. But the direction is unambiguous: a photo is no worse than what you were doing before. It is worse only than a kitchen scale.

Why 20 per cent error is good enough

The thing that is easy to miss: running a diet does not need an accurate number, it needs a repeatable one.

If your counter systematically under-reports by 15 per cent but does it consistently, you will still see that the week you ate more shows a higher number. You will still see that when you hold your target, the scale moves. A constant error cancels out the moment you start comparing days with each other instead of comparing each one against a table.

A simple rule falls out of this: calibrate your calorie target to your own counter, not to theory. A formula gives you a starting point. Your real target is the number at which your weight does what you want it to — after two or three weeks of watching.

At that point the question “was that dinner 620 or 740 kcal” stops mattering. What matters is whether today looked like yesterday, and whether the week looked like the target.

How to shoot so the estimate lands closer

A handful of habits noticeably narrows the spread and costs nothing but a second of attention:

• Shoot at an angle, not from overhead. A frame from around 45 degrees shows the height of the food, not just its outline. Top-down throws away all the volume information.
• Leave a reference point in frame. The whole plate, a fork beside it, a hand at the edge. Anything of known size.
• Photograph before eating. A disturbed portion is harder to estimate than an intact one.
• Type in what cannot be seen. Two tablespoons of oil, cream in the soup, sugar in the coffee — this is the one piece of information no photo can recover.
• Separate the components where you can. Three things side by side estimate better than one mixture of them.
• Daylight, or strong light from above. Shadows blur the edges of a portion and take away the contours the model works from.

When to reach for the scale instead

There are situations where a photo is simply the wrong tool, and it is worth knowing them rather than assuming the app will cope with everything:

• Soups and sauces — the ratio of water to everything else cannot be read off the surface.
• Cooking oils and fats — invisible, and often the single largest item in the meal.
• Nuts, seeds and nut butters — very calorie dense, so a 20 g error is a 120 kcal error.
• Alcohol — the calories are not visible in the colour of the liquid, and drinks differ widely.
• Contest prep or a therapeutic diet — anywhere precision matters more than trend.

A sensible arrangement in practice: a photo day to day, the scale where the stakes are high. You can also weigh a few of the meals you eat most often once, remember the result and keep using photos afterwards — calibrating your own typical portions takes one evening and then works for months.

This article is informational and is not medical advice. The formulas below are estimates for healthy adults. If you have a medical condition, take medication, are pregnant or are breastfeeding, set your targets with a clinician rather than a calculator.