Search for a command to run...
Marcus is a long-time Java and web developer. He's always curious to learn new technologies and excited to share his insights with others. He is the VP of Developer Relations at Vaadin and an international speaker who has presented at over 100 events worldwide.
No comments yet. Be the first to comment.
Proving business value remains a long-standing challenge for DevRel teams and leaders. I've heard anecdotes of DevRel professionals spending over half their working hours justifying their roles. That's clearly not great. This lack of trust can become...
As a frequent conference speaker, I was curious to know what the hot topics are in the Java ecosystem. To find out, I analyzed accepted talks at four large Java conferences held in 2023: JFokus, DevNexus, Devoxx UK, and JCON Europe. The most popular ...
Generative AI tools like ChatGPT have transformed the landscape of product development seemingly overnight. They've made it easier and faster to convert ideas into products. But as tools help us write more code quicker, we need to keep our focus on b...
Using vector embeddings for context
On this page
It's remarkable how quickly you can go from an idea to production with the help of AI tools.
Earlier this week, I had a thought: what could I learn about trending topics in Java if I analyzed accepted talks at Java conferences? Thanks to ChatGPT and Copilot, I was able to create the analysis and publish my Java Ecosystem Trends Report 2023 less than a day after getting the idea.
I had all the skills needed to write the scraper, categorize data, and analyze the result. What I didn't have was a week to do it all by hand. Working with AI tools, I could translate my idea into reality in a fraction of the time.
Here's what I wanted to do: find out which topics were most frequently present in accepted conference sessions.
I had a rough idea of the steps I needed to take:
Scrape conference data from websites
Categorize the data into distinct topics
Analyze the data to find out the most common topics and any other interesting data
Write a report
I began by asking ChatGPT to write me a web scraper script in Node based on what I was trying to accomplish. I went back and forth between coding in my IDE to asking ChatGPT to improve certain aspects of the script until I had something that worked for all the conferences I wanted to analyze. The output was a JSON file with session titles and abstracts per conference.
Next, I asked ChatGPT to help me write a Node script that takes the JSON file as input and sends each title+abstract to the OpenAI chat completion API to get classified. Again, I had to go back and forth several times. First, to work on the prompt, then to batch the requests and show a progress indicator as the process took a long time. I used the new OpenAI function calling mechanism to ensure I got valid JSON as output. The output was a CSV file with 3 columns: conference, topic, and count.
I performed the analysis step in a Jupyter Notebook using Python, Pandas, and Seaborn. I worked with ChatGPT to quickly do the analysis and create charts. I liked having the Python code allowed me to verify the work (as opposed to using a more automated AI data analysis tool).
With the analysis done, the last remaining part was drawing conclusions and writing up a report. I already had some takeaways, but I still asked ChatGPT to look at the list of most common topics to see if it could develop other insights. Most of the insights were aligned with what I was already thinking, and others weren't that valuable. But some of its insights helped spark more ideas for me.
Finally, I wrote the report and hit publish. The 2023 Java Ecosystem Trends Report was born. I purposefully published it early as an MVP instead of trying to make it perfect. It has allowed med to get great feedback and insights on how to make the categorization more robust for next time.
Know what you're trying to accomplish before you start. If not, you'll end up on a wild goose chase.
Work in small steps. Give enough context and iterate until you get the result you need.
Verify the results. Just because something is a statistically probable answer to your question doesn't mean it's right. You should be able to stand behind what you create.
Ship early and gather feedback.