Everyone’s talking about AI agents—few have actually built them. Even fewer have made them work. So, what does it really take to move from hype to execution? In this episode of Innovation Tales, Alexandre Nevski sits down with Konrad Jeleń—Executive CTO at PMR, VP of Data Science & AI at Kolomolo, and Director for Technical Solutions at AI4Process. With 20+ years of experience in AI and enterprise transformation, he’s seen what works, what fails, and what businesses need to get right.

Key Takeaways

  • AI agents are evolving: Businesses are shifting from AI as an assistant to AI as a process participant, capable of automating workflows and making decisions.
  • Specialization improves AI performance: Dividing AI into specialized agents, each with a specific role, enhances efficiency and accuracy in business operations.
  • Governance is essential: Without proper oversight, AI agents can introduce risks; balancing automation with human control is key to success.

Register for the upcoming live webinar on AI Agents for Business: https://innovation-tales.com/event/ai-agents-for-business/

Meet the Expert: Konrad Jeleń

Headshot of Konrad Jeleń, Executive CTO at PMR and AI expert.
Konrad Jeleń, AI and enterprise transformation leader, shares insights on AI agent adoption.

Konrad Jeleń is a physicist with a background in computational and experimental physics, specializing in AI, data science, and enterprise architecture. With nearly two decades of experience, he has developed advanced models in forecasting, vision AI, anomaly detection, and generative AI, helping organizations harness the power of intelligent automation.

Currently, Konrad serves as Executive CTO at PMR Limited, VP for AI and Data Science at Kolomolo LTD, and Director for Technical Solutions at AI4Process LTD. Previously, as Technical Director at Pegasystems, he led AI integration, enterprise architecture, and BPM initiatives, driving innovation in situation and risk management.

Beyond corporate leadership, Konrad has played a pivotal role in establishing and scaling Centers of Excellence, designing specialized technical services that enable organizations to deploy AI-driven BPM solutions effectively. His consulting work spans advising Saudi ministries and key EU agencies, including the European Aviation Safety Agency (EASA), on AI strategy, enterprise architecture, and data science.

An entrepreneur at heart, Konrad has launched multiple startups, with his latest ventures focusing on legal case processing solutions powered by advanced AI—enhancing efficiency and analytical capabilities for legal firms.

Renowned for his pragmatic approach, Konrad believes in prioritizing stability, efficiency, and purpose-driven AI solutions over chasing the latest trends. His methodology ensures that AI technologies align with real-world business needs, delivering reliable and impactful results.

Defining AI Agents

AI has already become an integral part of many workplaces, assisting employees in drafting emails, summarizing reports, and brainstorming ideas. But the real game-changer lies in AI agents that can move beyond passive assistance to active participation in business processes. These agents don’t just provide information—they collaborate, make decisions, and automate workflows. The big question is: how do organizations integrate AI agents effectively without introducing unnecessary risks?

Unlike traditional AI models that generate responses based on prompts, AI agents operate more like specialized subject matter experts, each with a defined role. Konrad explains that an AI agentic system functions like a team of specialists, with different agents focusing on specific tasks, such as:

  • Mathematical Reasoning: AI models that excel at complex calculations and logic-based problem-solving.
  • Code Generation: Agents designed to suggest and validate code snippets.
  • Tool Utilization: AI that can interact with external tools, such as search engines or internal enterprise systems.
  • Process Automation: Agents capable of executing actions, such as booking flights or processing customer returns.

“I typically try to get them to feel like they have a quorum of people, typically experts in some fields, that would be talking about the subject matter and trying to find a solution to the problem that was given to them to solve.”

By combining these specialized agents into a collaborative framework, businesses can significantly enhance operational efficiency. However, this approach requires careful structuring to ensure seamless interactions and prevent unexpected outcomes.

Real-World Applications of AI Agents

AI agent systems are already making an impact across various industries. Konrad shares a compelling example from a recent AI hackathon, where a team built an agentic system for customer service. The system used multiple AI agents to manage customer inquiries efficiently:

  • Intent Classification Agent: Identifies whether the request is about returns, technical support, or general inquiries.
  • Sentiment Analysis Agent: Evaluates customer emotions to ensure an appropriate response.
  • Product Information Agent: Gathers relevant details from databases, forums, and documentation.
  • Return Processing Agent: Handles logistical aspects such as shipping details and refund policies.
  • Wrap-Up Agent: Concludes the conversation and solicits customer feedback.

This structured approach not only improves customer experience but also optimizes response times and reduces operational costs. However, implementing such systems requires balancing automation with human oversight.

Challenges and Safeguards in AI Systems

“We’ve actually built an agentic system that looks at the inquiry made by a customer chatting, and it would classify this intent into one of the predefined intents and further engage on an agentic chat that was specifically attuned to this kind of intent.”

While AI agents offer significant advantages, they also introduce challenges that businesses must address. One of the most pressing concerns is ensuring that AI systems do not make costly mistakes. For example, an AI-powered booking system could mistakenly purchase expensive flights without proper authorization.

To mitigate these risks, companies need to implement several safeguards:

  • Fine-Tuning AI Models: Training AI agents with extensive datasets to minimize errors.
  • Standard Operating Procedures (SOPs): Establishing clear guidelines for agent behavior.
  • Governance Agents: AI supervisors that monitor and correct agent interactions.
  • Human Oversight: Ensuring critical decisions require human intervention.

By combining these strategies, businesses can harness the power of AI agents while maintaining control and minimizing risks.

The Cost Factor: Balancing AI Investment and ROI

One of the biggest hurdles in AI adoption is the cost of development and deployment. AI systems require substantial investment in infrastructure, training, and ongoing maintenance. Organizations must weigh these costs against potential benefits.

Key cost considerations include:

  • Development Costs: Designing, training, and fine-tuning AI models.
  • Operational Costs: Running AI models, especially those that rely on cloud-based APIs.
  • Scalability: Expanding AI capabilities without exponentially increasing expenses.
  • Risk Mitigation: Implementing safeguards to prevent financial losses due to AI errors.

Companies should start with small, domain-specific AI implementations before scaling up. By leveraging techniques like model distillation and small language models, businesses can optimize performance while keeping costs manageable.

“You would need to find out what the metric is for your human-based system that is important to you and the metric that you would like to achieve.”

AI Governance: Maintaining Control Without Stifling Innovation

AI governance is crucial to ensuring that AI agents operate within predefined ethical and operational boundaries. Transparency and accountability are key factors in preventing AI-related mishaps. Businesses should implement:

  • Continuous Monitoring: Tracking AI agent interactions to detect anomalies.
  • Audit Trails: Maintaining logs of AI decisions for accountability.
  • Ethical AI Frameworks: Aligning AI behavior with company values and regulatory requirements.
  • Adaptive Learning: Using real-world data to improve AI performance over time.

By striking a balance between automation and oversight, organizations can build AI systems that drive efficiency while maintaining trust.

Alexandre Nevski and Konrad Jeleń discussing AI and digital transformation.
Host Alexandre Nevski and guest Konrad Jeleń discuss AI agents and enterprise transformation

The Future of AI Agents in Business

“The AI systems are going to get better. And, of course, the end result is going to be that those chats are going to be better.”

AI agents will continue to evolve, becoming more autonomous and integrated into enterprise workflows. However, businesses must remain vigilant in assessing AI’s impact on operations and customer interactions. The key to success lies in:

  • Defining clear objectives for AI adoption.
  • Prioritizing governance and ethical considerations.
  • Investing in AI literacy among employees.
  • Continuously refining AI models based on real-world performance.

As AI technology advances, companies that approach AI adoption strategically—focusing on measurable business value rather than hype—will be best positioned for long-term success.

Konrad’s Recommendations

Agentic AI

  • Question if agents are necessary – A single LLM may be simpler, easier to control, and more efficient. Use agentic AI only if there’s a clear advantage.
  • Minimize complexity and cost – Agentic AI is expensive due to internal communication overhead. Keep agent clusters small and use smaller, domain-specific, distilled, or fine-tuned models to ensure efficiency and task relevance.
  • Avoid large agentic workflow automation – Current agent-based workflows are fragile and hard to control. Limit agents to well-defined, manageable tasks.
  • Implement strong governance – A dedicated governance agent should monitor outputs, catching mistakes and dangers before they lead to serious consequences.
  • Measure agent performance with data science-driven benchmarks – AI outcomes must be validated against predefined quantitative metrics, ensuring results align with project goals. Compare against baseline models, such as traditional algorithms or simpler ML approaches, to verify if agentic AI truly provides an advantage.

General AI

  • Define a measurable goal – AI must solve a specific problem with clear success criteria.
  • Ensure high-quality data with ground truth – Without the right data, AI is useless.
  • Use a baseline model first – Test AI’s value against a simple reference model.
  • Smarter, not bigger – The simplest effective model is often the best choice.
  • Measure success early – Define key metrics and track AI’s real-world impact.

Conclusion

AI agents are rapidly transitioning from experimental tools to essential components of enterprise automation. As businesses explore their potential, success hinges on a careful balance between efficiency, oversight, and cost-effectiveness. Konrad Jeleń’s insights underscore the importance of specialization, governance, and iterative improvement to ensure AI-driven solutions create real value. By thoughtfully implementing AI agents, organizations can enhance operations while mitigating risks, setting the stage for a more intelligent and responsive business environment.

Join the Conversation

Want to dive deeper into AI agent adoption and governance? Connect with Konrad Jeleń on LinkedIn to explore more expert insights.

What Do You Think?

As AI agents become more autonomous, how can businesses strike the right balance between automation and human oversight? Share your thoughts with the Innovation Tales community on LinkedIn. And don’t forget to register for the upcoming live webinar: AI Agents for Business!

Further Reading

Innovation Tales
Innovation Tales
Redefining Business Processes: The Power of AI Agents with Konrad Jeleń
Loading
/

Episode timeline:

  • 00:00 Introduction: The Future of AI in Business
  • 00:00 Meet the Expert: Konrad Jeleń
  • 00:00 Defining AI Agents
  • 00:00 Real-World Applications of AI Agents
  • 00:00 Challenges and Safeguards in AI Systems
  • 00:00 Cost Considerations in AI Implementation
  • 00:00 Practical Advice and Future Outlook
  • 00:00 Conclusion and Final Thoughts
Title
.