AI Engineering & LLMs
Tools, frameworks, and challenges in building AI and LLM-based applications.
Working with AI Engineering / LLM Tools
It's split,43% are building LLM apps, but 57% haven't started yet. Still early days, so lots of room to grow.
AI/LLM Systems in Production
41% have 2-5 systems in production,moving past experiments. But 21% have nothing in production yet, and another 21% only have one. Most teams are still figuring out how to scale.
Role of AI Engineering Tools
40% use AI tools for critical production stuff, while 37% are still experimenting. About a quarter use them regularly but not for anything mission-critical. The industry is transitioning from experiments to real production.
Challenges in AI/LLM Systems
79% are worried about evaluation and reliability,can you trust what the AI outputs? Integration (66%) and cost (55%) are also big headaches. You need good tech, money, and organizational buy-in to make AI work.
For which use cases do you currently employ AI/LLMs-based applications? (Select all that apply)
Document processing is the most popular use case (71%),LLMs are great at extracting info from messy docs. Knowledge base Q&A (65%) shows RAG is becoming standard. Customer support (53%) and code generation (47%) are also common. Companies are moving from simple stuff to more complex agentic workflows (41%) and autonomous agents (32%) as they get more experience.
Which managed LLM services or cloud-based providers do you use? (Select all that apply)
OpenAI is the clear winner at 62%,ChatGPT's API is basically the default. But 18% are self-hosting, which is interesting. AWS Bedrock (15%), Anthropic (12%), and Google (12%) are also popular. Lots of companies use multiple providers to avoid vendor lock-in.
Do you self-host open-source models? (Select all that apply)
59% don't self-host,managed services are just easier. Among those who do, vLLM is popular (18%) for efficient inference. Some build custom stacks (12%). Self-hosting is mainly for companies that need privacy, want to save money at scale, or need custom models.
Which AI application patterns do you use? (Select all that apply)
76% start with simple prompts,easiest way to get started. RAG (71%) is the next step for connecting LLMs to your own data. Tool calling (35%) and multi-step workflows (35%) are more advanced. Fine-tuning (29%) and hybrid systems (21%) are for the pros. There's a clear progression from simple to complex.
Which frameworks or libraries do you use to build or orchestrate AI applications? (Select all that apply)
LangChain is the most popular at 56%,it's basically the standard. But 34% don't use any frameworks, either building custom stuff or keeping it simple. Some companies build their own (19%) for competitive reasons or specific needs. The field is still figuring out what works best.
Do you use any of the following vector databases for LLM-powered applications? (Select all that apply)
28% don't use vector databases,probably doing simple embeddings or skipping RAG. Elasticsearch leads at 28% (people using what they already have). Chroma, pgvector, and Qdrant are all around 19%,no clear winner. Pinecone (16%) is popular for managed services. People pick based on what they already have, scale needs, or specific features.
How do you generate or manage embeddings? (Select all that apply)
44% use open-source models,cheaper and models like sentence-transformers work well. Managed APIs (28%) are easier, and 28% mix both approaches. 13% aren't using embeddings at all. Depends on scale, budget, and what you need.
Do you evaluate or test AI/LLM outputs systematically?
38% do manual evaluation, but 31% don't evaluate at all,that's risky for production. Only 17% have automated evaluation. This matches the fact that 79% say evaluation is their biggest challenge,people know it's a problem but haven't solved it yet. We need better tools and practices.
Do you use any tools to monitor AI/LLM systems in production? (Select all that apply)
48% use custom monitoring, but 41% don't monitor at all,that's dangerous for production. Specialized tools like LangSmith (10%) and Evidently AI (10%) aren't widely adopted yet. Most people build custom stuff because existing tools don't handle LLM-specific things like prompt performance or token usage.
Where do you run AI / LLM workloads? (Select all that apply)
47% use cloud-managed services,easiest option. 33% run custom stuff in the cloud for more control. 30% are on-premise for privacy, compliance, or cost reasons. 10% do hybrid. Depends on what your company needs.
How do you access or provision GPUs for training/fine-tuning or running LLMs?
57% use cloud GPUs,easiest way to get started. 24% have their own GPUs, usually for cost savings at scale or data privacy. 14% are CPU-only, probably using smaller models. Specialized inference providers (5%) are still niche. Most people prefer managed solutions because GPUs are complicated.
Do you have a dedicated GenAI/LLM team in your organization?
56% don't have dedicated AI teams,most AI work happens in existing data or engineering teams. The 44% with dedicated teams are probably bigger companies or ones that really prioritize AI. The field is still new, so companies are figuring out the best structure.
How would you describe your AI engineering maturity?
53% are in early production with basic standards, and 27% are still experimenting,most companies are still learning. Only 10% have solid monitoring and evaluation practices, and another 10% have advanced platform capabilities. The field is moving fast, and most teams are still building the basics.
Which AI engineering tools or technologies do you plan to adopt or expand in the next 12 months?
People are planning to focus on monitoring and observability,they know there are gaps. Fine-tuning and autonomous agents are also on the roadmap. Plans are pretty diverse (each around 7%), which makes sense since teams are at different stages. The focus on observability matches the fact that evaluation and monitoring are top concerns.