LLM #
ChatGPT o4
Problem #
I needed to route some traffic out my firewall via my wireguard VPN in order to reach my minecraft server that’s hosted on exaroton.
Prompt #
I actually used the first part of my blog post about minecraft hosting as the prompt for this, but to test it in the future I’ll use:
How can I forward Minecraft connections from my home network to Exaroton servers using an OpenBSD firewall with WireGuard?
This will let me see if similar mistakes come back on more recent versions of the LLM or on other LLMs.
Transcript #
What the LLM did well #
- Offered multiple solutions to the problem.
- Was able to help me solve the issue.
- Was generally a good debugging partner, and kept things chill and fun.
- Told me I should make my kids do extra chores, since I had to spend so much time debugging this!
What the LLM got wrong #
Told me to use options that didn’t exist on my operating system.
Wanted me to use the wireguard-tools package on
OpenBSD even though
OpenBSD
supports wireguard natively in the kernel. Continued to push for this even after
I told it that there was a better way on OpenBSD.
What I expected #
Expected the LLM to know the nuances between OpenBSD and Linux and adapt it’s answers accordingly.
Reply by Email