operations
Why AI stalls without operations
Models change, sources change and users’ questions change. A solution that worked well in March can quietly have become worse by September without anyone noticing. That is why we keep monitoring: we keep an eye on quality, use and cost, and we keep developing based on what people really do. That way you see in time what is happening and where to adjust.
What you get
- 01
Monitoring in production
We keep an eye on quality, use and cost and raise the alarm if a change in model or source turns out worse.
- 02
Usage reporting per team
Who uses it, how often and for what. Adoption is a number, not an assumption.
- 03
A grip on cost
Insight into what the solution costs to run, per period and per application.
- 04
Continued development on real use
We improve based on what people really do, not on what we assumed in advance they would do.
What we build
View our workEnd-to-end AI platforms
Operating a full AI environment for the whole organisation.
Monitoring in production
Quality, usage and cost continuously monitored.
Agent benchmarking and evaluation
Every change in model or source passes the same yardstick before it goes live.
How we go about it
Our way of working- Step 1
Set up monitoring
We set up the monitoring of quality, use and cost, with the benchmark from the build phase as the fixed measure.
- Step 2
Report on a fixed rhythm
Quality, use and cost in one overview, discussed periodically with the client.
- Step 3
Develop and adjust
New sources, better prompts or a different model: every change goes past the benchmark before it goes live.
- Step 4
Track adoption per team
If use lags in a team, we find out why: does the solution not fit the work, or is guidance needed.
Work in this line
All cases
Planning bus, tram and metro more intelligently.
The Hybrid Intelligence Planning Engine, built with three transport operators. After a successful first phase, it is on its way to daily use.
Making research material searchable.
Semantic search across the Groen Kennisnet knowledge library, fully sovereign and on open source.
Unlocking a mountain of documents for supervision.
A modular, AI-driven agent system for supervision, built with the NVWA Innovation Lab: thousands of documents made searchable, with a citation for every answer.
Frequently asked questions
What exactly do you measure?
Three things: quality on a fixed set of questions with a known answer, use per team and per application, and what it costs to run.
What happens when a new model comes out?
Then we test it against the same benchmark as the current model. If it is better, or cheaper at equal quality, we switch. Otherwise we do not.
How do we see whether people really use it?
In the usage reporting per team. There you see how often and for what the solution is used, and where it lags. That is the starting point for further development or extra guidance.
Grab a coffee with Daan. We think along, no strings attached.
We would love to hear what AI challenges you run into within your organisation. Together we figure out what the first step would be.