Building In-House Generative AI Expertise Through Hands-On Agentic RAG Bot Development

       Date Published：08 JULY 2026

Nippon Totor Co.,Ltd. provides information systems and fan service systems to public racing venues across Japan. The department responsible for supplying information terminals felt it first needed to build up in-house expertise before it could consider new services powered by generative AI.

Rather than outsourcing a finished solution, the team wanted to work hands-on and understand both the possibilities and limitations of generative AI. In response, Classmethod provided four months of hands-on support for a proof of concept (PoC) to build a RAG-powered AI bot.

Learning How Generative AI Works, Not Just How to Use It

The Nippon Totor team behind this project is responsible for deploying and maintaining systems such as the information inquiry terminals used by users of the company’s services, as well as overseeing the underlying infrastructure.

Their first real encounter with generative AI came while planning a new service. While considering a system or service built on generative AI, they realized they did not yet have the expertise to judge how it should fit into the business.

“Our knowledge of generative AI was so limited that it would have been difficult to start using it in the business on our own,” Mr. Y said. “So our goal was to catch up on what we lacked by building things ourselves, with the help of a hands-on support service.”

What Nippon Totor wanted was not a finished product built by an outside vendor. It wanted to build the skills to design, build, and improve systems itself, while keeping the resulting knowledge in-house. Of the several companies it consulted, Classmethod was the one that understood that intent most concretely.

“Other companies mostly proposed building it for us, but Classmethod respected our wish to do the work ourselves,” Mr. Y said. “I felt they presented the most concrete answer to what we were actually looking for.”

That set the stage for a four-month project built around learning the fundamentals of generative AI and running a technical proof of concept.

From Lectures to Implementation: Building AWS and Generative AI Expertise

The project began with lecture-based sessions on the basics: what generative AI is, and what RAG is. From there, the team built an environment to run generative AI on AWS and began testing an AI chatbot that answered questions from internal documents, aimed at making inquiry handling more efficient.

Nippon Totor took the lead in doing the hands-on work, with Classmethod stepping in wherever the team ran into difficulty. In addition to a regular meeting about once a week, the team consulted with Classmethod as needed through a chat tool, repeating cycles of building the development environment, loading data, and checking answer accuracy.

Classmethod’s support went well beyond simple lectures. The team built GenU, an open-source generative AI chat application published by AWS, on top of Amazon Bedrock, using Anthropic’s Claude as the model. Classmethod supported everything from environment setup to operations needed for the proof of concept, including the process for loading documents into Amazon Bedrock Knowledge Bases, IAM user management, and a development and operations workflow built around GitHub.

In practice, though, the team ran into a real challenge: they needed AWS knowledge and generative AI knowledge at the same time. The company had experience with AWS, but this department did not work with it on a day-to-day basis.

“Because generative AI sits on top of AWS, we had to understand both how the AI worked and how AWS itself was built, at the same time,” Mr. M said. “It was tough going at first, but Classmethod walked us through the detailed settings carefully, and little by little I started to feel like I could build things myself.”

Following the instructions step by step still produced errors. Investigating the cause did not always lead to a fix. Each time a problem came up, the team consulted Classmethod, worked through it with Classmethod’s advice, and resolved issues one at a time. Through this steady back-and-forth, the team gradually came to understand the structure of AWS and generative AI.

“As the work progressed, our understanding of development tools like Git also improved,” Mr. Y said. “With AWS too, we started to see where different settings were located, and I could feel our knowledge deepening.”

Rather than leaving everything to Classmethod, the team tried things themselves, ran into setbacks, and moved forward by asking questions along the way. That process itself became an important part of how the department came to understand generative AI.

Surfacing Tacit Knowledge and Organizing Documentation: Lessons from Building an AI Bot

When it came to loading real business documents into the AI chatbot, a major challenge emerged: much of the company’s know-how had never been written down. Even with specifications, manuals, and design documents in hand, material written for a human reader is not always interpreted the way the team intends when an AI reads it. Revision histories and references to past specifications sometimes affected answer accuracy, pointing to a new challenge: using AI well means organizing not just documents, but the knowledge itself.

“Every time we tried to build a sample answer, the conversation turned to which document we’d need to produce that answer,” Mr. M said. “But in reality, there was a lot of tacit knowledge that wasn’t written down anywhere. AI can’t answer based on information that hasn’t been put into text, so I felt the volume of documentation the chatbot would need was far bigger than we imagined.”

To address this, Classmethod also helped tune the system to improve the answer accuracy of the RAG system.

The team tested reranking to re-evaluate search results, query decomposition to break complex questions into multiple search queries, metadata filtering to narrow the search scope, and rephrasing questions. They also converted documents with MarkItDown and revisited the parsing settings in Knowledge Bases, reshaping the business documents into a form the AI could work with more easily.

In the second half of the project, the team also tested an Agentic RAG approach built on Amazon Bedrock AgentCore. By adding a mechanism where the AI worked out on its own what information it needed for an answer, then searched again to fill any gaps, the answers came much closer to what the team had hoped for.

“We compared the same questions, and answer accuracy improved dramatically with the Agentic RAG approach,” Mr. WS said. “The answers started coming much closer to what we wanted, so since then we’ve mainly used the Agentic RAG approach built on Amazon Bedrock AgentCore.”

Of course, the AI cannot immediately answer the way a veteran employee would. Even so, there was real value in the AI presenting supporting source material and cutting down the time spent tracking down what to check.

“It’s not giving us fully veteran-level answers, but it feels like it’s reached the level of a motivated third-year employee,” Mr. Y said. “It’s a huge help that, through back-and-forth conversation, it can work hard and pull together an answer of decent quality. It feels like we’ve gained one more team member we can rely on.”

Looking Ahead: Expanding Generative AI to Venue Operations and the Back Office

What the four-month project produced was not just the AI chatbot itself. Sharpening the team’s understanding of what generative AI can and where it falls short, and what kind of preparation and design it takes to bring it into day-to-day work, was itself a major achievement.

Going forward, the team plans to keep improving the accuracy of the QA system and use it to make inquiry handling and document searches more efficient. Beyond that, the team is also considering applying the system to providing information to fans of public racing and similar events and supporting the back-office operations of the municipalities and organizations that run them.

“In the near term, we’re aiming to raise the accuracy of the QA system to a level that comes close to an experienced team member’s answers,” Mr. WT said. “Longer term, I think the wave of generative AI will eventually reach fan services and the back-office side of the municipalities that run these venues, so I’d like to deepen our expertise further so we can be the ones providing support.”

Younger team members also spoke about new possibilities for AI-powered information delivery. Mr. SM and Mr. ST each shared their outlook on the future, particularly around new kinds of race prediction material AI could make possible.

“For example, we might be able to offer data analyzed from video and audio as new prediction material that fans want,” Mr. ST said. “It would be reassuring to have Classmethod’s support going forward as we think through what this new kind of information could look like.”

“This proof of concept broadened our perspective on what kind of possibilities open up when our company puts AI to use,” Mr. SM said.

As a provider of various services, Nippon Totor says it will keep exploring what new, generative AI-powered services might look like. Where the company cannot handle something entirely on its own, it sees continued collaboration with Classmethod, its partner on this project, as one of its leading options.

“This time, we received extremely thorough and attentive support,” Mr. Y said. “When we consider using generative AI for our next service, I expect Classmethod will be the first company we turn to.”

From a vague awareness of generative AI to being able to map out how to put it to work in the business, Nippon Totor has taken its next step, building toward new services that support the venues and organizations behind Japan’s public racing.