Behind the Scenes: Building the AI-First Nation (AIFN) Knowledge Assistant – LLX
In this article, I will share the evolution of the AIFN chatbot now named as LLX, which stands for Laurence Liew eXtended. Think of LLX as a virtual twin and eXtension of myself. LLX can advise anyone in the world 24x7x365 who is keen to learn about the AIFN book contents and/or AI Singapore’s programmes.
One of my motivation for writing “AI-First Nation: A blueprint for polic makers and organisation leaders (AIFN)” was to highlight AI’s potential as a personal assistant for boosting productivity. I have always lived by the “walking the talk” principle, so I wrote a book using AI to understand its benefits for myself. The end goal was the AIFN book written and published in 3 months. I could never have achieved that without the help of AI.
The book led me to create a website, and then I thought it would be cool to have a chatbot so people could interact with and “chat” with the AIFN book.
From Simple to Sophisticated: The Evolution
My initial implementation in August 2024 leveraged Google Gemini’s impressive large context window to build the chatbot. I loaded the entire book into the system prompt and had a functional chatbot running in an afternoon. This worked remarkably well – Gemini could handle the full context and provide accurate, nuanced responses with no need for complex Retrieval Augmented Generation (RAG) systems.
The success with Gemini, especially its ability to handle my book’s content within its context window, became the foundation for the first version of the AIFN chatbot. However, my website has regular blog articles, and I thought it would be a waste not to include these articles into the chat. Instead of constantly changing the system prompt, I chose a longer-term solution using established LLM-RAG techniques.
These first versions of the AIFN chatbots and the current LLX chatbot are all built on WordPress with the chatbot functionality provided by MeowApps’s AI Engine. I strongly encourage everyone who is keen to build a web-based chatbot on WordPress to check out AI Engine. A user-friendly WordPress plugin that gives you access to leading LLM APIs and even your own local LLMs via Ollama.
At this time, I had three chatbots on the AIFN website. One to chat with the AIFN book, one to do the AI Readiness Index (AIRI) assessment via a chat interface, and a new personal AIRI assessment to help an individual understand his/her own AI maturity and how to start or accelerate their own AI learning.
At various times, these three chatbots could be using OpenAI GPT4o, 4o-mini or Google Gemini Flash as its backend LLM engine, depending on the flavour of the week and which responses performed best. I found this to be less than ideal as a user would have to interact with 3 different “personalities”.
A single expert is usually the preferred point of contact for most people. So I have been experimenting with combining all 3 system prompts and knowledge base into a single chatbot.
I call this Laurence Liew eXtended or LLX, a play on my name the popular term LLM. I believe LLX accurately reflects the concept, as the LLX chatbot represents an extended version of myself in the digital space.
As of this first week of December 2024, LLX is powered by Anthropic’s Claude 3.5 Sonnet. I’ve found its responses to be consistently superior, and a casual poll amongst my colleagues revealed a shared preference for Claude’s output.
Beyond Simple Q&A
AIFN’s LLX can engage users in meaningful conversations about AI Singapore’s programs and the AI-First Nation book content. It can:
- Provide detailed insights into programs like 100E and AIAP
- Conduct complete AI Readiness Index (AIRI) assessment for organisation and help the organisation the next steps to take to move from say AI Unaware to AI Aware.
- Conduct complete Personal AI Readiness Index (pAIRI) assessment for the individual and provide a learning journey to improve him or herself.
- Guide users through complex topics around their AI journey
Technical Implementation Insights
One fascinating discovery was how efficiently large language models could handle multiple complex knowledge domains in a single unified prompt. Rather than creating separate specialized bots for the book (AIFN), the AI Readiness Index (AIRI), and the personal AIRI (pAIRI) for an individual, I found that a well-crafted system prompt could address all three domains.
This streamlined my implementation while maintaining high-quality responses across different topics.
To ensure consistent, high-quality outputs, I implemented the usual best practices:
- Structured formatting guidelines for different response types
- Example-guided responses for maintaining quality
- Specific confidence thresholds for response handling
An interesting aspect of the implementation was using the LLM (Anthropic’s Claude 3.5 Sonnet in this case) itself to improve its own outputs. If an LLM’s output was unsatisfactory (like a poorly formatted report), I’d paste the image back into it, asking for a critique and a revised format, along with a better system prompt.
Core Design Principles
Several key principles guided the design of the system prompt.
- Clear Persona Definition
- Expert advisor embodying knowledge from the book
- Professional yet approachable tone
- Clear boundaries about being an AI assistant
- Ability to guide users through assessments like AIRI and pAIRI
- Contextual Awareness
- Group behaviors with clear segmentation in the system prompt
- Included 3 articles I wrote before ChatGPT era in the system prompt to help LLX adopt my style and tone.
- Used RAG to embed the whole AI-First Nation book content
- The system pull AIFN Insights newsletter articles regularly into the same RAG vector database (Pinecone vector database).
- Google search integration to pull in latest information from curated websites
- Since LLX is present on both AIFN’s homepage and article sidebars, we used a system prompt that lets LLX know it is operating within WordPress and it should change its behaviour accordingly.
- Response Framework
- Concept of short vs long format criteria
- Explicit instructions for AIRI and pAIRI structured report formatting
- Multiple choice question formatting for AIRI and pAIRI
- Visual appeal and readability guidelines
- Ability to infer assessment questions from rubrics (this was a new recent discovery – just 2-3 days ago).
The Prompt Taught Me “how to fish”
The original AIRI chatbot included the 12 questions we asked in AIRI as part of its system prompt.
When building the Personal AI Readiness Index (pAIRI), I provided the original AIRI system prompt including specific context of what I wanted pAIRI for and generated the pAIRI system prompt. I then tweaked the pAIRI prompt further and asked a few users to test it. Everyone liked the way the questions were asked, how sensible the MCQ options were, and how the final recommendation report provided enough guidance for someone to start their AI learning journey.
Then 3 days ago, one of my users and myself were discussing the outputs of the chatbots and it got me reviewing in detail both the AIRI and pAIRI system prompt side by side. Then, I discovered there were no 12 questions for pAIRI in the system prompt. The whole QUESTIONS section was missing, but the pAIRI chatbot was still functioning well with the right questions asked and MCQ options offered made sense.
Though I had a hunch about how it worked, I wanted a simple explanation for my team, so I turned to Claude and asked it why pAIRI worked without the 12 questions like in AIRI:
The pAIRI prompt was effective despite not having pre-defined questions because it provided a robust structural framework that enabled me to:
1. Understand the Assessment Purpose:
- The prompt clearly defined the 5 pillars and indicated there should be 12 dimensions
- It provided a clear example question with multiple-choice format
- It explained how answers should map to AI readiness levels (Unaware, Aware, Ready, Competent)
2. Follow Clear Report Guidelines:
- Detailed instructions for analyzing responses
- Framework for generating recommendations
- Specific guidance on tone and approach
- Clear structure for the final report format
3. Use Program Alignment:
- Clear mapping of recommendations to AI Singapore programs (AI4E, AIAP, etc.)
- Guidelines for matching programs to maturity levels
- Framework for suggesting learning pathways
4. Leverage the Example:
The sample question about "Individual Strategy & Culture" served as a powerful template, demonstrating:
- Question structure
- Response option format
- How to map choices to readiness levels
- Appropriate depth and complexity
So while having the 12 pre-defined questions (like in AIRI) would provide more consistency, the prompt's strong structural guidance and clear example allowed me to generate appropriate questions aligned with the assessment's goals. This worked because the focus was more on the framework and outcome (measuring AI readiness and providing recommendations) rather than specific questions.
The prompt essentially taught me "how to fish" rather than giving me "the fish" - allowing me to generate contextually appropriate questions while maintaining the assessment's integrity and purpose.With the above insights, the current version of LLX – does not have the hard-coded 12 questions for both AIRI and pAIRI anymore!
What We Learned and How to Do It Right
Key lessons from this implementation:
- Prompt Engineering
- Using LLMs to improve their own outputs
- Importance of example-guided formatting
- Value of unified prompts for multiple domains
- Power of well-structured system prompts
- Technical Choices
- Initial simple implementation can be highly effective
- RAG adds flexibility for content updates
- When deciding between cost and capability, consider both factors carefully (OpenAP GPT4o-mini was notably cheaper than Anthropic’s Claude 3.5 Sonnet, but users preferred Claude’s outputs).
- Importance of scalable architecture
- Quality Control
- Example-based formatting ensures consistency
- Built-in confidence thresholds maintain quality
- Clear response frameworks improve user experience
- Regular monitoring and refinement of outputs
Conclusion
The LLX (the AIFN Knowledge Assistant) continues to evolve as I learn new techniques or when the backend LLM API provider introduces new capabilities. This shows that building a sophisticated AI chatbot today is accessible to non-programmers (I did not need to code a single line of Python or any code to build this chatbot).
LLX is not just a chatbot, it is a real-world example of the “AI-First” approach from my book. It is about “walking the talk” – I cannot expect companies and professionals to use AI every day if I do not.
What is next? I am looking into how a more agentic approach can improve LLX’s performance and effectiveness.
Would you like to try LLX yourself? Visit aifn.sg and start a conversation! Your interactions might just help shape the future of AI in Singapore and beyond.
