JVM’s Open Source All-Stars, June 2023: LLMs Edition - JVM Weekly vol. 48
If you think Java hasn't gotten on the hype train associated with LLM - you're thinking wrong. That's why today we're going to take a look after new open-source that are just getting proper traction.
I've been thinking lately about how to make our editions more interesting, and I've been trying on the idea that the last one of each month should be dedicated to interesting JVM Open-Source projects that have appeared (or started trending, because that is often the case) in the previous 30 days. However, I'm not referring to prominent Open-Source projects like Micronaut or Smallrye, but more to the lesser-known gems in our community. Since the holiday, as always, are a somewhat quieter period, I decided now is a good time to do that. Interestingly, this edition has turned out to be rather thematic, which adds an unexpected twist.
In 2023, Large Language Models (LLMs) are the talk of the town, and even the JVM ecosystem, not typically connected to this field, has been caught between these waves of hype. Today, I bring you four projects, each with some tie-in to language models. Additionally, we will delve into the underlying concepts upon which each featured library is built.
VertexAI-kt
While OpenAI and ChatGPT often dominate public consciousness, other players also offer their unique solutions. Google's Vertex AI is a case in point. It's a comprehensive machine learning platform that's been under development for years (remember when Google was synonymous with "The AI Company?), and it now comes enhanced with support for LLMs and generative artificial intelligence. This platform includes advanced models like PaLM 2 (OpenAI's GPT equivalent), Imagen (the Text-to-Image model), and Codey, which is Google's response to GitHub's Copilot. GitLab, for instance, shared how it employs Codey as an example for its "explain this vulnerability" feature.
hexascribe/vertexai-kt, on the other hand, is a Kotlin Multiplatform library specifically engineered to streamline the integration with the Vertex AI API, all tailored for Kotlin. Although VertexAI-kt primarily caters to mobile applications, its common segment should be applicable to virtually any project, including those on the server-side. After all, MultiPlatform means MultiPlatform, not MultiMobileOS.
But if you're building your own mobile apps, the same organization in its repositories also has a set of Kotlin MultiPlatform-based components for easily creating OpenAI-based chatbots - hexascribe/chatbot-builder.
langchain4j
LangChain, a framework developed by Harrison Chase, caters to the needs of Large Language Models (LLMs). It surged in popularity in the wake of the immense success achieved by OpenAI's GPT-3, and arguably even more with GPT-3.5 and GPT-4. Launched at the end of October 2022, LangChain leverages these advancements, providing a platform specifically designed for creating production-level applications that rely on LLMs, rather than just experimental ones.
LangChain's key idea is to link or "daisy-chain" different parts together to make complex applications. These chains can use pieces from many modules like templates for various prompts (like chat responses, ELI5, etc.), connections with different LLM models (such as GPT-3, BLOOM, etc.), agents (which use LLMs to decide what to do — we'll discuss the gpt-engineer project in this category later today), and memory (short- and long-term). All of this is offered in the form of a standard application framework.
langchain4j/langchain4j, in turn, is a Java wrapper for langchain - one on steroids. The whole thing already leads users by the hand and focuses on clearly showing the usage possibilities (in the project you will find a given directory with examples), and the creators announce further development. At this point, the whole thing hasn't yet broken through to mass consciousness, but I'm personally keeping my fingers crossed - and since new commits have appeared even as I'm writing this, and the creators got an invitation to Devoxx Belgium in Issues, I think we'll still hear about langchain4j.
midjourney-proxy
While the preceding two projects might not ring a bell, it's safe to say that anyone who hasn't been entirely out of touch for a year is likely familiar with Midjourney. There are numerous LLMs for producing images... but Midjourney stands among them. Your mileage might vary, but in my perspective, neither the Stable Diffusion nor the Dalle project can hold a candle to Midjourney - even a novice like myself can accomplish remarkably impressive outcomes.
Nevertheless, Midjourney does have a limitation - it lacks an API. The sole interface to interact with this project is Discord, where a bot is used to generate images. But it's evident that unofficial ones tend to fill the void where official solutions are absent. This is where the third project for today, novicezk/midjourney-proxy, comes into play. It is a "fan-made" API, available both as source code and a Docker image. This requires an authorization key, obtainable from Discord's request headers (hackerman ⌨️), and facilitates image generation by offering the relevant endpoints and hookups. We can use these to check if an image has been created (Midjourney uses asynchronous communication).
However, just to add a bit of complexity, the repository, and all its documentation are in... Chinese. Luckily, it isn't overly complex, and navigating the Readme with DeepL provided me with the most amusement since the time I had to reverse-engineer the integration instructions of a $15 Asian smart-socket.
PS: I love Midjourney, it's the one that generates images for me for the post previews. And today's one I created using midjourney-proxy.
Chat2DB
Staying in the Chinese sphere of influence (albeit this time with English documentation), our next project is a SQL database client brought to life by Alibaba. However, true to the theme of this edition, it's not your mother's database client.
chat2db/Chat2DB includes integration with OpenAI (with the documentation even suggesting ways to connect from within China via VPN). This feature allows the conversion of natural language to SQL and vice versa, potentially boosting developers' productivity significantly. I confess that I've had the opportunity to experiment with generating SQL queries from within a ChatGPT-like interface, and I found it quite beneficial. Simultaneously, besides the API integration, the software also accommodates 'traditional' functions such as data management and supports various database types – MySQL, PostgreSQL, Oracle, and more – with prospects for further expansion.
Advanced features include an intelligent AI assistant that helps with query conversion and SQL optimization. In addition, Chat2DB supports teamwork, ensuring the security of an enterprise database account by eliminating the need for developers to know the online database password. The program supports multiple databases, including MySQL, PostgreSQL, Oracle, and others, with the possibility of future expansion.
Bonus
The final two repositories can be considered as a bonus - although they're not written in Java, they align well with this month's theme. So, if you're still eager for more after the first four projects, I invite you to continue reading till the end.
GPT Engineer
The AntonOsika/gpt-engineer is "an adaptable and extensible AI tool designed for programming tasks" - simply stated, Developer-as-a-Sevice. It generates code projects based on user instructions, allowing easy transfer of the initial specification (a text file in which you can also suggest what specialization you want the bot to assume) and subsequent interaction in the console, in case the bot needs clarification.
So, we're handling a somewhat more specialized variant of Auto-GPT – anyone familiar with the LLM field is bound to have come across this project. I'll be honest, I don't usually have much patience for these kinds of tools (although they do help foster empathy – one can imagine how frustrating it must be for project managers at times, struggling to articulate their objectives clearly). But here's a twist... without much effort, I was able to receive a simple scraper for open-source Java projects from GitHub. And it works correctly - I see it's going to be quite helpful for me next month
Microsoft's AI For Beginners
And for those who are not comfortable with what they have seen above and would like to explore the subject a little further due to FOMO, plus have some time - for them, there is a last project today, microsoft/AI-For-Beginners. This is because Microsoft employees have created a 12-week curriculum of 24 lessons focused on AI. The list of topics covers various approaches to AI, including symbolic AI, neural networks, and Deep Learning, using popular frameworks such as TensorFlow and PyTorch. Other aspects such as NLP, image analysis, genetic algorithms, and multi-agent systems are also included.
Also interestingly, the syllabus deliberately does not focus on practical applications, not wanting to shove specific solutions down the user's throat. However, for those who want to learn the theoretical underpinnings of modern AI approaches (classical, historical ones are deliberately omitted), the course will be an interesting resource.
Hope you enjoyed it 😄. Next week is the next edition (instead of Thursday/Friday, I now plan to publish after the weekend - on Tuesday) and this time we will return to the classic weekly review of multiple topics.
Finally, if you're very skeptical about AI, I recommend pessimistsarchive.org - it will show you how predictable our every reaction to new technology is.
