agnos.is

Published: 2025-04-17T22:27:07+02:00

I have created a new filter that enhances LLM image understanding by looking up location names from image data, when available.

It is a simple filter that pre-processes user messages and looks up the address/location name of the most recent user-supplied image. By providing the LLM with the GPS coordinates embedded in an image, along with the address (including country), the LLM can answer questions like “where was this picture taken?” accurately, reason about things located nearby where a photo was taken, and more.

It reduces hallucinations by telling the LLM where photos were taken, or instructing it to inform the user that it's not possible to determine location when no EXIF GPS data can be found.

This filter is based on my work for the OSM tool.

For the next iteration of this filter, I want to do the following:

• Configurable location granularity. Less granular = better caching, at the expense of location accuracy.
• Read other EXIF data (altitude, etc).
• Handle multiple images better.

Getting and Configuring the Filter

🗺️ You can read the instructions for the filter here.

You can install the tool from the Open WebUI website.

Or you can install it directly from my git repository.

License: CC-BY-SA-4.0.

Written by: @[email protected]