A graph containing structured knowledge from greater than 120 articles on mathematics and data science
In this text, I’ll show how you possibly can structure and explore the content of your personal articles using graph technology and a few programming.
The thought of using NLP techniques for structuring unstructured data will not be latest, nevertheless, the most recent progress in LLMs (Large Language Models) has sparked countless opportunities for doing just that. The accessibility for amateurs through the booming technology Chat-GPT has created lots of attention towards LLMs and generator models.
In actual fact, generative AI is on the agenda in lots of firms already!
The way in which we’ll work with the technology in this text is thru the programming language Python using OpenAI’s developer API. We’ll work on data from Medium (meta huh?) and construct a knowledge graph. Which will sound like a mouthful, but it surely is definitely surprisingly easy to start with.
First things first. The plan of attack is the next.
- Get the API to work and access it through Python.
- Use a sample text to do prompt engineering ensuring that the GPT-4 model understands what you would like from it.
- Download your articles from Medium (you possibly can in fact use other pieces of text in case you want) and pre-process the info.
- Extract and collect output from Chat-GPT.
- Post-process the output from Chat-GPT
- Write code to structure the info further right into a graph using the Cypher query language.
- Mess around along with your latest best friend and explore your articles.
Without further ado, let’s start by quickly establishing the fundamental tech.
We’d like to have the programming language Python and the graph database Neo4j installed on our local computer.
The very first thing to do is to be certain that you may have a plus account at OpenAI so that you could use GPT-4. The…