A more semantic approach to chunking of text documents for embedding to be used for chat completion with OpenAI LLMs.

Update: Working with other OpenAI developers, we've come up with a way to automate this process even further using the LLMs themselves. Discussed here: [ Ссылка ]

By the way, the chunk header idea at 07:29 is not mines. I got it from www.BlinkData.com who provide a ChatPDF service.

00:00 Introduction
01:19 The Issue
03:39 The Solution
05:10 Real World Example
09:04 How To?
10:49 Conclusion

свернуть