Building the “AI Boardroom”: My Journey with Multi-Agent Orchestration
We’ve all experienced the limits of a single AI prompt. You ask for a complex report, and the AI misses the latest news, loses the tone halfway through, or simply forgets the original goal. In the professional world, we solve this with teams. You wouldn’t ask a lead researcher to write the marketing copy, and you wouldn’t ask the copywriter to verify the technical data. You hire specialists and a manager to coordinate them.
Here’s how I built a system that researches, writes, and reviews — all while I have the role of the “Grand CEO” and am still in the lead.
1. The Core Architecture: Hierarchical vs. Sequential
Most AI workflows are sequential. But in a real company, you have a manager. I implemented a Hierarchical Process where a “Project Manager” (The Boss) oversees the workers.
In this model, the Manager is the only one who talks to the user. The workers (Researcher, Writer, Librarian) are “siloed.” They don’t talk to each other; they only report to the Boss.
Why this matters:
- Reduced Hallucinations: Agents stay focused on their specific niche
- Context Control: The Manager decides exactly what information is relevant for each agent to see
- Scalability: You can add more specialists without breaking your workflow logic
- Parallel Processing: The Boss can delegate multiple tasks simultaneously
2. Technical Secret: Tiered Intelligence
One of the biggest mistakes in AI development is using the most expensive model for every task. It’s like hiring a Senior Architect to change a lightbulb.
Here I implemented Tiered LLMs – as an example:
- Gemini 2.5 Pro: Assigned to the Boss for high-reasoning, complex orchestration, and quality control
- Gemini 2.5 Flash: Assigned to the workers for lightning-fast execution and lower token costs
Why use a “super-brain” for a simple search? Efficiency is key. In my testing, this approach reduced API costs drastically, while still maintaining high output quality.
3. The Specialized Team: Three Agents, Three Purposes
Here’s where tool specialization pays off – each agent has access to different capabilities:
- The Web Scout: Armed with real-time internet search
- The Research Librarian: Has access to your document library and PDFs
- The Writer: Focused purely on content creationThis separation mirrors a real research team.
The Web Scout finds breaking news, the Librarian cross-references with historical data and saved documents, and the Writer synthesizes everything into a cohesive narrative.
4. Chronological Awareness
AI often lives in a “timeless” bubble. By injecting a dynamic timestamp into the agents’ backstories at runtime, my agents always know exactly what day it is, ensuring their “latest news” is actually from today.
import datetime
now = datetime.datetime.now().strftime("%A, %d. %B %Y")This gets injected into each agent’s backstory and this simple addition solved a massive problem: my agents no longer reference “recent events” from 2023 when it’s actually 2026.
5. The Code in Action
Here’s the expanded architecture:python
import os
import datetime
from crewai import Agent, Task, Crew, Process, LLM
from crewai.tools import tool
from crewai_tools import FileReadTool, PDFSearchTool
from duckduckgo_search import DDGS
now = datetime.datetime.now().strftime("%A, %d. %B %Y")
print("\n--- Welcome to the Multi-Agent Research Command ---")
user_objective = input("Boss, what specific task should the team work on today? ")
print(f"Understood. Starting research on: '{user_objective}'\n")
boss_llm = LLM(model="gemini/gemini-2.5-pro")
worker_llm = LLM(model="gemini/gemini-2.5-flash")
os.environ["OPENAI_API_KEY"] = "NA" # Set to NA to prevent accidental use of OpenAI models
@tool("internet_search")
def internet_search(query: str):
"""Searches the internet for real-time data using DuckDuckGo."""
with DDGS() as ddgs:
results = ddgs.text(query, max_results=5)
return [r for r in results]
file_tool = FileReadTool()
pdf_tool = PDFSearchTool(
config={
"llm": {
"provider": "google",
"config": {
"model": "gemini/gemini-2.5-flash",
},
},
"embedder": {
"provider": "google",
"config": {
"model": "models/embedding-001",
"task_type": "retrieval_document",
},
},
}
)
knowledge_dir = 'knowledge'
if not os.path.exists(knowledge_dir):
os.makedirs(knowledge_dir)
print(f"? Created '{knowledge_dir}' folder. Add your files there!")
# The Library for the Librarian
all_files = os.listdir(knowledge_dir)
text_files = [os.path.join(knowledge_dir, f) for f in all_files if f.endswith('.txt')]
pdf_files = [os.path.join(knowledge_dir, f) for f in all_files if f.endswith('.pdf')]
# Initialize sources only if files exist
knowledge_sources = []
if text_files:
from crewai.knowledge.source.text_file_knowledge_source import TextFileKnowledgeSource
knowledge_sources.append(TextFileKnowledgeSource(file_paths=text_files))
if pdf_files:
from crewai.knowledge.source.pdf_knowledge_source import PDFKnowledgeSource
knowledge_sources.append(PDFKnowledgeSource(file_paths=pdf_files))
# Agent 1: The Web Scout
web_researcher = Agent(
role="Web Intelligence Scout",
goal="Identify the most significant AI breakthroughs from online sources.",
backstory=f"You are a data-driven researcher who monitors the web. Today is {now}.",
tools=[internet_search],
llm=worker_llm,
allow_delegation=False,
verbose=True
)
# Agent 2: The Research Librarian
librarian = Agent(
role="Research Librarian",
goal="Cross-reference findings with existing documents and research papers.",
backstory=f"Meticulous librarian with archive access. Today is {now}.",
tools=[file_tool, pdf_tool],
llm=worker_llm,
allow_delegation=False,
verbose=True
)
# Agent 3: The Writer
writer = Agent(
role="Content Strategist",
goal="Transform research into engaging, accurate content.",
backstory=f"Skilled writer creating compelling narratives. Today is {now}.",
llm=worker_llm,
allow_delegation=False,
verbose=True
)
# The Manager (Boss)
boss = Agent(
role="Strategic Manager",
goal=f"Lead the team to produce a comprehensive research report on {user_objective}.",
backstory=f"Orchestrator delegating to Scout, Librarian, and Writer. Today is {now}.",
llm=boss_llm,
allow_delegation=True, # Manager MUST have this to coordinate
verbose=True
)
# Master Task
master_task = Task(
description="""Research the latest developments on {user_objective},
cross-reference with our document library, and produce a 500-word article.""",
expected_output="A well-researched, engaging article ready for publication.",
human_input=True,
agent=boss
)
# The Crew Assembly
crew = Crew(
agents=[web_researcher, librarian, writer],
tasks=[master_task],
manager_agent=boss,
process=Process.hierarchical,
verbose=True,
max_rpm=2
)
# Execute
result = crew.kickoff()
print(result)6. Why the Librarian Changes Everything
Adding the Research Librarian solved a critical gap in my workflow. The Web Scout finds current information, but the Librarian provides:
- Depth not matched by pure internet research: Access to research papers, whitepapers, and internal documents
- Fact-Checking: Cross-referencing claims against trusted sources in your library
- Competitive Intelligence: Analyzing saved competitor reports and market research
- Historical Context: “This idea was already tried in 2022, here’s why it failed…”
In practice, this three-agent setup creates a 360-degree research view: current trends (Web Scout) + institutional knowledge (Librarian) + compelling narrative (Writer) = comprehensive output.
7. Human-in-the-Loop: The Safety Net
The most powerful feature, however? human_input=True.
Before the Boss finishes the task, she stops and asks me if the result is satisfactory. It’s the perfect blend of automation and human oversight. When the work is done, the program pauses. The Boss presents the draft and asks for feedback. You can say “Looks good!” or “Add more technical depth” or “The Librarian missed our Q4 report. Check that too.”
The Boss will coordinate the changes with the relevant specialists and return for a second review.
This creates an iterative refinement loop that’s impossible with single-agent systems.
But this orchestration only really works if you, as the human at the helm, know the “why” behind the “what.” In a multi-agent setup, deep subject matter expertise and a firm grasp of the underlying theory are the engine that allows you to be the Alpha AI Leader, as also laid out in a previous article ‘Alpha AI: The New Alpha Leader Culture‘. You set the strategic and cultural tone that these agents follow, turning an iterative refinement loop into a high-impact business asset.
8. Real-World Application: From Prototype to Production
When I first built this multi-agent system, this was just the groundwork. Once I validated the multi-agent approach, I saw a real opportunity: App Store Optimization is the perfect use case for AI orchestration. ASO requires multiple specialized skills: keyword research, competitive analysis, creative optimization, and performance monitoring, that no single AI can master simultaneously.
As such I built what I call an “AIgency”: a production-ready system that orchestrates 5 specialized AI agents, each with distinct personalities, expertise, and tools:
- Liza, our SEO & Social Media Expert
- Konstantin is the Brand Detective an Researcher
- Sky focusses on Trend-Intelligence and Algorithms
- Sam is our Tech Guru and Performance Specialist
- and finally Echo, the Writer
Now, this system isn’t just faster, but it’s smarter, too. Because the agents are specialists with distinct perspectives, they catch what I might miss. They work in parallel, spot patterns across domains, and challenge each other’s assumptions. The Manager orchestrates their collaboration, ensuring their outputs complement rather than conflict.
My AIgency obviously grew a bit more complex over time and by now spans a few thousands lines of code – but this really is AI orchestration in production: not replacing human expertise, but amplifying it through specialized, coordinated AI agents working as a team.
Key Takeaways
Building this system taught me that the future of AI isn’t just “better prompts” – it’s better management:
- Role Clarity: Give your agents a specific “Why” and “How”
- Hierarchy: Let a smart model manage the fast models
- Tool Specialization: Match tools to expertise (web search ≠ document analysis)
- Complementary Skills: Build teams where agents’ strengths cover each other’s weaknesses
- Oversight: Always keep a human in the loop for the final 5% of quality
CTC.rocks
Curious it all works together? Visit the full crew at ctc.rocks. (still “Work in Progress, and currently only in German, but you’ll get the idea)