# The real power of AI: getting from idea to reality quicker

It's remarkable how quickly you can go from an idea to production with the help of AI tools.

Earlier this week, I had a thought: what could I learn about trending topics in Java if I analyzed accepted talks at Java conferences? Thanks to [ChatGPT](https://chat.openai.com/chat) and [Copilot](https://github.com/features/copilot), I was able to create the analysis and publish my [Java Ecosystem Trends Report 2023](https://marcushellberg.dev/java-ecosystem-trends-report-2023) less than a day after getting the idea.

I had all the skills needed to write the scraper, categorize data, and analyze the result. What I didn't have was a week to do it all by hand. Working with AI tools, I could translate my idea into reality in a fraction of the time.

## Here's how I used AI tools to ship my idea faster

Here's what I wanted to do: find out which topics were most frequently present in accepted conference sessions.

I had a rough idea of the steps I needed to take:

1. Scrape conference data from websites
    
2. Categorize the data into distinct topics
    
3. Analyze the data to find out the most common topics and any other interesting data
    
4. Write a report
    

![](https://cdn.hashnode.com/res/hashnode/image/upload/v1688149861499/6ccffee9-c34a-4ebd-81f1-309daa43fc4c.jpeg align="center")

### Scrape data

I began by asking ChatGPT to write me a web scraper script in Node based on what I was trying to accomplish. I went back and forth between coding in my IDE to asking ChatGPT to improve certain aspects of the script until I had something that worked for all the conferences I wanted to analyze. The output was a JSON file with session titles and abstracts per conference.

### Categorize data

Next, I asked ChatGPT to help me write a Node script that takes the JSON file as input and sends each title+abstract to the OpenAI chat completion API to get classified. Again, I had to go back and forth several times. First, to work on the prompt, then to batch the requests and show a progress indicator as the process took a long time. I used the new [OpenAI function calling](https://platform.openai.com/docs/guides/gpt/function-calling) mechanism to [ensure I got valid JSON as output](https://yonom.substack.com/p/native-json-output-from-gpt-4). The output was a CSV file with 3 columns: `conference`, `topic`, and `count`.

### Analyze data

I performed the analysis step in a Jupyter Notebook using Python, Pandas, and Seaborn. I worked with ChatGPT to quickly do the analysis and create charts. I liked having the Python code allowed me to verify the work (as opposed to using a more automated AI data analysis tool).

### Write the report

With the analysis done, the last remaining part was drawing conclusions and writing up a report. I already had some takeaways, but I still asked ChatGPT to look at the list of most common topics to see if it could develop other insights. Most of the insights were aligned with what I was already thinking, and others weren't that valuable. But some of its insights helped spark more ideas for me.

Finally, I wrote the report and hit publish. The [2023 Java Ecosystem Trends Report](https://marcushellberg.dev/java-ecosystem-trends-report-2023) was born. I purposefully published it early as an [MVP](https://en.wikipedia.org/wiki/Minimum_viable_product) instead of trying to make it perfect. It has allowed med to get great feedback and insights on how to make the categorization more robust for next time.

## Key takeaways

* Know what you're trying to accomplish before you start. If not, you'll end up on a wild goose chase.
    
* Work in small steps. Give enough context and iterate until you get the result you need.
    
* Verify the results. Just because something is a statistically probable answer to your question doesn't mean it's right. You should be able to stand behind what you create.
    
* Ship early and gather feedback.