22. August 2024

AI Chat Meets Document Search

Generative AI chatbots like ChatGPT and their underlying Large Language Models (LLMs) have taken the world by storm, with their powerful capabilities to understand and generate human language text. Despite this hype, the field is still exploring the best ways to leverage LLMs productively while avoiding their problematic tendency to produce hallucinatory outputs.

We primarily investigate the benefit of AI chatbots in conjunction with a widespread problem: searching for documents or relevant information hidden within them. We have implemented numerous customer projects in this area, leading to the development of our HIBU platform. This platform already includes countless features and enables the rapid and cost-effective development of custom search solutions.

A seamless integration of chatbot and search engine combines and extends the individual strengths of both systems, as demonstrated by the following use cases we have implemented:

  • Users search for documents (or content elements), open one of the search results in an integrated document viewer, and use the chatbot to ask questions about that document. All of this happens within the same UI without any context switching.

  • Users search for documents and then use the chat function to ask questions across all search results. Again, the user remains within the search UI and does not have to deal with context switches.
    With search terms and domain-specific search filters, the user can narrow down the set of search results with ease and accuracy such that the chat engine looks for answers only within the relevant documents.

  • Users chat with the entire document repository. In this use case, the chat functionality effectively also serves as the search engine, giving the user great flexibility in meeting their information needs.

All these use cases rely on Retrieval-Augmented Generation (RAG). This method ensures that the chat engine does not invent answers. With this approach, the LLM can fully leverage its strengths in understanding and generating language, but it is not expected to also provide the relevant expert knowledge. Instead, this knowledge is found within the relevant documents.

In each of these cases, our chat functionality is enriched with a range of convenience features that boost user experience and productivity. For example, chat responses are linked to the most relevant document sections supporting these answers. This is a substantial step towards “explainable AI” (XAI).

The HIBU Chat in Practice

  • Get the best of both worlds: Search and Chat
  • Run HIBU either in the cloud or on-premise.
  • Choose wether to use an LLM by an external service (e.g., OpenAI) or a self-hosted LLM (data privacy!).
  • Either way, HIBU integrates chat in a cost-effective way (minimizing data traffic with the LLM).

Karakun – Your AI Partner

For running AI systems productively and generating value with them, AI knowledge alone is not enough. Suitable knowhow in engineering, DevOps, language technology and others are also critical. Karakuns offers all these competences in combination.

And, despite the hype, chat is not the best AI option for all use cases.

Let us consult you as to how AI can help to automate some of your business processes and support your knowledge workers in their daily tasks.

Contact us by Mail
give us a call
7. June 2023

HIBU @ SwissText 2023

From June 12 to 14, 2023, text analysis experts from industry and academia will meet at SwissText 2023 at the University of Applied Sciences Western Switzerland in Neuchâtel. In addition to our commitment as a Gold Sponsor, we will be moderating the “Battle for NLP Ideas” for the first time. This is a collaborative event where participants group together to brainstorm innovative ideas based on the latest NLP technologies. The best ideas are presented on stage and shared with the whole conference.

Efficient processing of information requests with PID Cockpit

When the revised Swiss Data Protection Act (revDSG) comes into force in September 2023, companies must have implemented technical and organizational measures to protect personal data following the new legal requirements. One of these processes is the response to requests for information from individuals. For this purpose, we have developed the PID Cockpit, a solution that offers users enormous time and cost savings.

This AI-based software solution lets users efficiently identify personal data in heterogeneous IT environments. For this purpose, PID Cockpit can search all data sources whose files can be accessed (directly or via API). It is irrelevant whether the data is structured or unstructured.

Poster

Our Language Analytics & Search Solutions expert Dr Holger Keibel will present the PID Cockpit to interested visitors during the poster session. In addition, visitors can get comprehensive information about our HIBU platform at our booth in the attached exhibition. HIBU is a flexible software platform for the cost-effective development of customer solutions, especially in the areas of intelligent search solutions, content analytics and workflow automation.

Download Flyer
Get in Touch with us
11. November 2022

Software for efficient data protection

The revised Swiss Data Protection Act (DSG) will become effective on September 1, 2023. By this date, companies must have implemented technical and organizational measures to protect personal data following the new legal requirements. Violations could result in severe penalties.

In addition, there is an increasing risk of cyber attacks in which unprotected or inaccurately stored customer and employee data are exploited. That leads to reputational damage and, in the worst case, to the loss of annoyed customers.

Together with our partner krm, we wrote the following article about why software solutions are essential for effectively and efficiently implementing data protection measures. This article was published in issue 11/2022 of Swiss IT Magazine.

Preview article

Identifying personal data in heterogeneous IT environments

One such application for automating manual processes is our award-winning PID Cockpit. The „PID Cockpit“ is a joint development between Karakun and the Swiss Information Governance Competence Centre (krm.swiss).

This AI-based software solution lets users efficiently identify personal data in heterogeneous IT environments. For this purpose, the PID Cockpit can search all data sources whose files can be accessed (directly or via API). It is irrelevant whether the data is structured or unstructured. The system recognizes industry- and company-specific text types when set up correctly. It can be run on local servers or in the cloud, as required.

The benefits for users are simplicity and the associated time and cost savings. Automated, transparent AI processes also minimize errors and make search results reproducible and consistent.

Would you like to learn more about the PID Cockpit? We would be happy to advise you. You can get a first overview in our product flyer.

E-Mail us
Call us
download Flyer
18. October 2022

AI-SDV 2022

AI-SDV is the place to be for everyone involved in advanced search and data applications, text mining and visualization technologies. AI-SDV 2022 took place on Oct 10-11, 2022, in Vienna (Austria).

Karakun expert Dr. Holger Keibel presented a talk about information extraction from tabular documents.

Abstract & Slides

In our customer projects involving automated document processing, we often encounter document types providing crucial data in the form of tables. While established text analytics algorithms are usually optimized to operate on running text, they tend to produce rather poor results on tables as they do not capture the non-sequential relations inside them (e.g. interpret the content of a table cell relative to its column title, interpret line breaks inside a cell differently from line breaks between cells or rows).

While there are elaborate information extraction products in the market for a few highly specific types of tabular documents, there is no general approach out there. The main cause for this is the fact that table structures can be encoded by a heterogenous range of layout means (e.g. column boundaries can be signaled by lines vs. aligned text vs. white space).

In this talk, we will illustrate several solutions that we have developed for a range of challenges occurring in this context, both for scanned and digitally generated documents.

Want to learn more about how language analytics and information extraction can boost you business?

Get in touch with us
3. October 2022

PID Cockpit wins 2nd place

After being nominated for the sol.IT-Award 2022, we were happy to have another highlight a couple of days later:

At the official award ceremony at the Bits + Currywurst, we won second place with our PID Cockpit.

Konfettiregen bei der Preisverleihung des sol.IT-Awards 2022.

Claudine Zillmann and Markus Schlichting were on site for us at the Signal-Iduna-Park in Dortmund and were finally able to receive one of the coveted certificates.

Urkunde für den 2. Platz am sol.IT-Award 2022 - standesgemäss mit Pils und Currywurst...

We congratulate Frederik Dürr and the team of MyShoeFitter GmbH on the first place and are already looking forward to the next edition of this exciting competition..

PID Cockpit simplifies data protection tasks

According to national and international data protection laws, every person has the right to know what data is stored about him or her, in order to have it deleted or corrected. Since this information is nowadays stored in a wide variety of data pools, requests for information and deletion can make identifying personal data a real challenge. Data protection officers, governance & compliance specialists, and HR departments in SMEs and large enterprises need a simple yet effective solution.

In contrast to traditional approaches, the k2 PID Cockpit allows direct access to the data sets in question and the connection of a wide variety of data sources. It offers transparency in the identification of a wide variety of structured as well as unstructured data sources containing personal data.

Flyer downloaden
Informationen anfordern
7. July 2022

PID Cockpit launched

Together with the consulting company Kompetenzzentrum Records Management (krm), we launched the PID Cockpit for identifying and cleansing personal data in heterogeneous IT landscapes. The solution based on the text analysis platform HIBU uses automated AI-based processes for identifying and extracting personal data.

On September 1, 2023, the new Swiss data privacy law will take effect. Individuals will then have the right to know what personal identifiable data (PID) companies store about it. Upon request, companies have to delete or correct this data. Thus, data protection officers, governance & compliance specialists, and HR departments in SMEs and large companies should be well prepared.

Today’s heterogeneous data storage with different data pools can make requests for information, deletion, and the identification and accessibility of personal data, a real challenge. That’s where the PID Cockpit helps:

Unlike traditional approaches, it allows direct access to affected data pools and the integration of various data sources. It offers transparency in identifying a wide range of structured and unstructured data sources that contain personal data.

By automatically locating documents and records containing sensitive personal data, companies benefit from massive time and cost savings and comply with requests for information and deletion. (Elisabeth Maier, CEO Karakun)

The PID Cockpit builds on our text analysis platform HIBU, can easily be adapted to individual customer needs and can be integrated into a wide variety of IT infrastructures. The solution searches for personal data in standard and proprietary “home-grown” solutions as long as there is an application programming interface (API) to access the data sources. The system can recognize both company and industry-specific text types and language patterns. It can be hosted on-premises and in the cloud. Finally, the PID Cockpit can also build a basis for assessing whether data storage complies with legal requirements.

The benefits of the PID Cockpit are its consistency and reproducibility.
Automated AI-based processes replace monotonous and error-prone manual work. And the identification and extraction of information are objectified. The results of the PID Cockpit are insofar resilient, reproducible and consistent. (Bruno Wildhaber, CEO of krm)

We are happy to discuss your individual requirements. Give us a call at +41 61 551 36 00 or send us an email to hibu@karakun.com. Or download our product brochure for a first impression.

email us
call us
download brochure
20. May 2022

Karakun @ SwissText 2022

From June 8 to 10, 2022, text analysis experts from industry and academia meet at SwissText 2022 at SUSPI in Lugano. In addition to our commitment as a Gold-Sponsor, we will be presenting ourselves at the exhibition. We also actively contribute to the top-class conference program with the following talk:

Integrating ML-based Classifiers into an Enterprise Search System

HIBU is a proprietary software platform that we use to build customer solutions around enterprise search and multilingual text analysis. Its architecture provides two analysis pipelines: a first one embeds basic NLP steps, based on the detected document language and used to pre-elaborate the document’s content; a second one contains a sequence of high-level annotators, able to discover information in the document. Some examples are extracting entities from the text, such as persons, places and organizations, identifying paragraphs containing confidential information etc.

Both pipelines use the framework Apache UIMA to combine the annotators that are relevant for the target application. Each single one can be adapted and switched on and off by configuration. Moreover, the framework allows us to add new annotators based on the individual customer’s needs.

In this context, we recently integrated some new ML-based annotators as part of an Innosuisse project carried out in collaboration with SUPSI and DSwiss (“EXTRA”, presented separately, leveraging a fine-tuned version of the pre-trained BERT model and other ML technologies). These annotators allow us to provide scalable document classification, as well as customized information extraction, to be used by applications for further workflow-based functionalities.

In this demo we will show how we wrap the new functionalities into the base platform, and how these are integrated to further enrich the final results.

At our booth in the exhibition area, interested visitors can inform themselves about our HIBU platform. HIBU is a flexible software platform for the cost-effective development of customer solutions, especially in the areas of enterprise search, business intelligence and workflow automation.

Get in touch with us
28. June 2021

SwissText 2021

For the 6th time, the three-day online conference Swisstext took place from 14-16 June 2021. The conference is the meeting place for text analysis experts from Swiss business and research. It is organized by the association SwissNLP (in cooperation with the two universities FHNW and ZHAW). As regular participants, we were also involved this year with a contribution. The poster “Understanding structural information in scanned documents” shows our approach for analyzing and processing tabular information in scanned documents.

Information extraction from tables

With our HIBU-Plattform, we develop custom solutions in the areas of enterprise search and text analytics. In these projects, we often deal with scanned documents that are digitized using OCR software. Standard extraction methods work well for continuous text but generally deliver insufficient results for tables because they ignore the table layout and thus hide important information / relationships. Typical problem areas are table cells whose content breaks across multiple rows, especially cells whose content can only be interpreted using the associated column heading.

Poster

Do you have any questions about the poster or the procedure?. Dr. Holger Keibel will be happy to answer them.

Networking at an online conference? Check!

How can participants at an online conference effectively link up with interesting contacts? That was one of the questions we asked ourselves when offering the organizers of SwissText 2021 our help.

The result was a networking format in which each participant in a session was randomly sent into a virtual room together with three other people. Once there, each person had a limited amount of time for a personal introduction. Before the session automatically ended, participants could exchange contact details if interested.

Of course, such a format cannot replace face-to-face exchange on site. But within the possibilities of an online conference, it worked very well. As the feedback from the participants shows, it was worth taking part:

SwissText 2021 has just ended – and we are already looking forward to next year. Hopefully on-site – but certainly once again with active support from Karakun.

14. June 2021

Secure and retrieve sensitive data

More and more data is being digitised or produced directly in digital form. And more and more data and applications are moving to the cloud. Companies have to ask themselves how they can efficiently handle these volumes of data, evaluate them and find information in them. And how they deal with data that needs to be protected, in particular. How do you manage to offer automatic processing (AI) and other beneficial functions around data and reconcile these functions with data protection at the same time?

As part of the bridge2digital Security & Compliance event, we illustrated these challenges together with our partner DSwiss AG using current joint projects in the intersection of both companies. We took a closer look at search solutions and automatic document processing in the context of “zero-knowledge” data encryption.

Would you like to learn more about SecureSafe, the integrated intelligent search or the HIBU platform?

Case Story SecureSafe
zu Securesafe
16. November 2020

Karakun joins SwissNLP

Karakun AG is the latest member of SwissNLP.

SwissNLP aims to promote Natural Language Processing (NLP), Computational Linguistics and Text Analytics in Switzerland. The association brings together experts, solution providers and customers from industry and academia and organizes events and projects for NLP in Switzerland. SwissNLP is going to organize the upcoming Swiss Text Analytics Conferences.

AI (not only) for SME

Our in-house developed HIBU-Platform combines established and innovative technologies in the areas of enterprise search, language analytics and artificial intelligence (AI). Our customers benefit from rapid implementation of applications at budgetable costs.

Together with universities, we are validating new Deep Learning approaches that are ideally suited for SME as they normally have only a limited amount of data available.

Get in touch with us to discuss your individual project.

Get in touch with us


© Karakun AG, 2025. All rights reserved.