agnos.is

The agnos.is blog

Published: 2024-10-15T22:48:20+02:00

Short and simple update: two new filters that make use of Open-WebUI's new feature to render collapsible sections in order to separate the reasoning and thought processes from final output for models that support this kind of workflow.

There are two new filters: one for Reflection 70b and one for Artificium 8b.

These filters act on the final output of the models, and separate the LLM's reasoning and thinking steps from the final output, and put them into collapsible sections. It makes it easier to focus on the LLM's final answer. Additionally, the filters can optionally remove the thinking steps from the input when text is submitted to the LLM, in order to save on token use.

Artificium Thinking Filter Collapsible Thought Filter (Reflection 70b)

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-09-30T19:01:37+02:00

The 1.0.0 release of the OpenStreetMap tool for OpenWebUI has arrived!

https://openwebui.com/t/projectmoon/open_street_map/

The 1.0.0 release has the following changes that make the tool a powerful map searching solution OpenWebUI, allowing users to find points of interest (POIs) near any place: – Caching: all data except the actual POI search is now cached, reducing load on public OSM services. – Nameless entities are now handled. This is important for minor landmarks, playgrounds, and other leisure areas. – The tool now can now report its results as a citation in replies. – Breaking change: the Nominatim URL setting has changed.

Along with calculating travel distance to nearby destinations, and various other bug fixes over the past weeks, I feel like it's time for a 1.0 release.

Breaking Config Change

The biggest announcement is the breaking change of the Nominatim URL valve setting for the tool. This used to point directly to Nominatim's search endpoint. But now the setting must point to the root URL of the Nominatim instance, because it's now using two different Nominatim endpoints.

If you need to change the setting, your LLM will refuse to search OpenStreetMap and warn you about this.

Caching and Nameless Entities

The two most important changes are caching and nameless entity handling. The public OSM services are available for free to everyone, and the data does not change THAT often. By caching address lookup data, extra POI information, and navigation travel distance, a lot queries against the OSM servers can be avoided. This is particularly important for the common scenario of the user searching from their house, school, work: “What's the closest X near me?” Instead of hitting Nominatim every time to resolve the user's location, it is now simply pulled from the cache.

Additionally, many entities in OpenStreetMap do not have names. This is either because of lack of proper tagging, or the place may not actually have a name at all. This is very common with things like playgrounds in neighborhoods, small green spaces in cities, and so on. – The logic for determining what entities are “useful” to the tool has changed. – The tool now resolves an address for these nameless entities and uses it as the name.

Sending the results as a citation is just a nice bonus that gives some insight into what the OSM tool is doing when it searches for the user.

The Future

The tool is rapidly approaching feature completion. Releases in the immediate future are going to focus on: – Cleaning up the code for a maintainable future. – Fixing bugs and edge cases. – Adding more predefined POI search functions.

The code is a bit of a mess at the moment because of OpenWebUI's requirement to have all the Python code in one file. I am considering ways to deal with this, ranging from moving a lot of the code to a PyPI module, or simply just having a simple build file that concats the code together into one Python module.

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-09-27T23:29:52+02:00

Version 0.9.0 of the OpenStreetMap tool for Open WebUI integrates OpenRouteService (ORS) for more accurate results. Travel distances between locations are calculated more precisely, rather than relying solely on “as the crow flies” measurements.

Get the latest version of the tool

Haversine Distance

By default, the OSM tool uses a straightforward distance calculation method, known as “Haversine distance,” which can be misleading in real-world scenarios. The Haversine distance is the direct distance between two points on a sphere (Earth, in our case)..

For example, two points might be close by geographically but could take a significantly longer time to reach due to roads, traffic, and other factors. Integrating ORS helps overcome this by providing actual travel distances based on real road networks.

How to Enable ORS in the Tool

Go to the OpenRouteService website.

Sign up for a free ORS account and log in. Once logged in, navigate to the API token section and create a new token. This token will be used to authenticate your requests to ORS.

In the tool’s configuration settings, you’ll find field for the ORS API Key. Enter your API key here, and now the tool will use ORS routing to calculate more accurate distances!

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-09-25T21:51:41+02:00

I have updated the Checkpoint Summary Filter for Open WebUI to account for changes to Open WebUI's code. The filter reaches deep into the internals of Open WebUI to enable per-chat continuous summarization to handle the context limit, so with all of the recent refactorings in Open WebUI, this broke the Checkpoint Summary Filter. But I have patched it to run on the latest versions of Open WebUI (0.3.29+ at the moment).

In other news, I will delete the Narrative Memory Filter, as the Checkpoint Summarization Filter is meant to be its replacement, and there is no point in maintaining two filters. Not to mention, the Narrative Memory Filter's functionality is subpar.

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-09-23T22:16:34+02:00

In the latest versions of the OpenStreetMap tool for OpenWebUI (0.6.3+), I have added proper handling of real-time location. OpenWebUI has the ability to report the user's location via the browser geolocation API.

https://git.agnos.is/projectmoon/open-webui-filters

Previous versions of the OSM tool did not prompt the large language model (LLM) properly when this information was available, so results for questions like “Where is the nearest grocery store?” would either be completely nonsensical, or centered on some nearby major city that the LLM statistically associates with the GPS coordinates reported by the browser.

Proper Handling of Real-Time Location

In order to have the OSM tool be able to answer questions like “where is the nearest grocery store to me?“, it needs access to your realtime location. This can be accomplished with the following steps: – Enable user location in your user settings. – Create a model with a system prompt that references the variable {{USER_LOCATION}}. – OpenWebUI will automatically substitute the GPS coordinates reported by the browser into the model's system prompt on every message.

Without the {{USER_LOCATION}} variable in the system prompt, OpenWebUI will not be able to give your real-time location to the LLM, and the OpenStreetMap tool will not give accurate results.

How Does It Work?

This was quite simple to enable. OpenWebUI uses the docstrings of Python functions to assemble a “tools schema” for the language model to use. OSM tool prior to 0.6.3 generated a schema that told the LLM to search for things near an “address or place.” Because of this, models would dutifully give a place or address, and NOT GPS coordinates.

This resulted in interesting behavior. The model would pick a city that it statistically associates with the user's GPS coordinates, and feed that in as a place to search nearby.

A simple tweak fixes this. The schema now tells the model to search for a “place, address, or GPS coordinates.” Models will now give the OSM tool GPS coordinates when available, and thus the tool can now answer questions like “Where is the nearest grocery store?”

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-09-17T22:35:38+00:00

Two blog posts and two updates in one day. I have now just published 0.5.0 of the OpenStreetMap tool for OpenWbeUI.

Install it from OpenWebUI.com Get the code from agnos.is Git

The reason why this update is getting another blog post is because I added support for searching for Ways in OpenStreetMap, which makes search results much, much more accurate.

While many map features in OSM are nodes, or single points on the map, many others are defined as “ways,” or shapes on the map. Buildings, roads, areas (parks, parking lots, etc), and everything else that you can draw with a line is considered a “way.” In our case, we're mostly concerned with the buildings.

By changing the Overpass Turbo query, I was able to get the bounding information for Ways, which is basically the set of GPS coordinates that make up its shape. Calculate the center coordinate, and we can now add these Ways to the search results!

Of course, the 0.5.0 update includes everything from the 0.4.0 update:

  • Complete rewrite of search result handling to prevent incorrect OSM map links being generated, and bad info given.
  • New setting: Instruction Oriented Interpretation. Controls the result interpretation instructions given to the LLM.
  • The Instruction Oriented Interpretation setting is also a user valve that can be controlled on a per-chat basis.
  • Added ability to search for: public transit, schools/universities, libraries, bike rental locations, car rental locations.

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-09-17T20:22:27+00:00

I just published version 0.4.0 of the OpenStreetMap tool for OpenWebUI. This tool allows large language models (LLMs) to search OpenStreetMap to find points of interest near addresses and locations.

Install it from OpenWebUI.com Get the code from agnos.is Git

I rewrote the search result handling to ensure that links to OpenStreetMap are generated more accurately. I noticed that Mistral-Nemo was generating bad links: either the GPS coordinates would be the right numbers, but the wrong sign (negative or positive flipped), or the URL itself was invalid despite having the right GPS coordinates.

  • Result handling now generates a Markdown result list for increased accuracy.
  • Each entry includes a simple bullet list consisting of the GPS coordinates, address, etc.
  • The entry also includes a generated OSM map link.

A new setting called “Instruction Oriented Interpretation” has also been added. This setting controls the level of detail of the instructions sent to the language model. It should improve result consistency, especially with models like Hermes 3. You can also toggle the setting on a per-chat basis for more fine-grained control. Finally, the search functionality has been expanded to include public transit, schools and universities, libraries, bike rental locations, and car rental locations.

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-08-27T13:05:09+02:00

runit is being removed from Gentoo.

With the removal of runit from the Gentoo package repository, I have finally been forced to migrate to something new for managing user services on a system without systemd.

Gentoo's “default” init service is OpenRC, a modern alternative to the traditional SysV init (or other SysV-esque, like upstart) that was commonplace in most Linux distros up until ~2015, when systemd started taking over basically everywhere. Today, the landscape is very different:

  • Most distros use systemd. Debian, Ubuntu, Arch all use this by default, and usually ONLY support it, officially.
  • These distros have often been forked specifically to remove systemd, to varying degrees of success (Devuan, Artix).
  • There is a class of independent distros, like Void, that use other init systems.

Gentoo has, for a very long time, used OpenRC as its default init system. But Gentoo also has official support for systemd. Because of this dual support, the efforts of the Gentoo developers have resulted in major applications that would otherwise have hard systemd dependency being able to run on other init systems. Most notably, patches for the GNOME desktop are maintained by Gentoo developers to make it work with OpenRC.

User Services on OpenRC

One of the biggest missing features in OpenRC, for me, is user services. This is something systemd does very well. You just stick —user on a systemctl command, and you can run unprivileged services from your local user account.

User services are perfect for things like:

  • syncthing
  • rclone mounts
  • Protonmail Bridge

They're great for anything that needs to run all the time and stay out of your way.

https://wiki.gentoo.org/wiki/OpenRC/User_services OpenRC does not truly support user services. The best support for a “user service” with OpenRC is running a system-level service as a specific user. This is fine for basic single-user use cases, but still requires root access to modify or create new services. There are some other approaches in the wiki, but they are (in my opinion) even more hacky or complicated.

https://github.com/navi-desu/openrc/ Luckily, there is now a fork of OpenRC (called OpenRC-Navi) that adds user service support to OpenRC. The fork is available in Gentoo as sys-apps/openrc-navi. The original is available as sys-apps/openrc.

Using OpenRC-Navi User Services

Creating user services with the fork of OpenRC is straightforward; it's basically the same as creating system services.

Official User Service Guide OpenRC Service Script Guide

In summary:

  • Service scripts go in ~/.config/openrc/init.d/
  • Use the —user (or -U) parameter with rc-service, rc-update, etc.

Caveats

At the time of writing, here are some issues I ran into while porting my runit scripts to OpenRC scripts:

  • supervise-daemon does not currently work with user services. A permission denied error occurs.
  • Paths using ~/ (tilde for home) don't work correctly. I had to specify absolute paths to avoid some odd file not found errors.

Example

Here is an example user service for an rclone mount.

The service meant to be symlinked, one for each specific rclone mountpoint to be created. For example, if you have an rclone mountpoint called nextcloud:, then you would need to execute shell some commands like this:

ln -s ~/.config/openrc/init.d/rclone-mount ~/.config/openrc/init.d/rclone-mount.nextcloud
echo 'RCLONE_REMOTE=nextcloud:"' >> ~/.config/openrc/conf.d/rclone-mount.nextcloud
echo LOCAL_MOUNTPOINT=/mnt/nextcloud"' >> ~/.config/openrc/conf.d/rclone-mount.nextcloud
rc-update -U add rclone-mount.nextcloud default
rc-service -U rclone-mount.nextcloud start

Alternatives

Instead of using a fork of OpenRC, there are a few alternatives on a Gentoo system:

  • s6: Another init system that continues to be supported.
  • runit: runit might be saved by a proxy maintainerb and not removed.

Hopefully, the user service feature of OpenRC-Navi makes its way back upstream into the main version of OpenRC.

License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-08-09T21:05:33+02:00

There is a housing crisis everywhere in the Western world. I have noticed in each of the three countries I've lived in (USA, Iceland, The Netherlands). The specifics differ between countries, but the end result is always the same: lack of housing drives prices up rapidly, pricing most people out of the market. At what point will it break?

Iceland

Nowhere near enough new construction. Massive interest rates (~10% in 2024).

Netherlands

Nitrogen crisis. Government deadlock and bickering over unrelated issues. Nothing built. Attempting to solve problems by blaming external parties.

USA

Increasingly dystopian wealth inequality.

Filed under: culture, thoughts License: CC-BY-SA-4.0.

Written by: @[email protected]

Published: 2024-08-05T21:09:57+02:00

I have been working on a new project: AI filters and function-calling tools for Open-WebUI. Open-WebUI is an open source UI for Large Language Models, built on top of ollama, AUTOMATIC1111, and other leading open AI technologies.

You can find Open-WebUI here.

Large language models are all the rage right now. They are definitely overhyped, but they're also a paradigm shift in computing. Newer, more powerful, more accurate models come out every 3 months. They get better and better at automatically calling APIs automatically and informing their responses.

Filters and Tools in Open-WebUI enables a lot of ChatGPT-like functionality on top of what is already built in.

An OpenStreetMap tool is my biggest contribution so far.

The OSM tool is surprisingly accurate when used with the latest models like Llama 3.1 and Mistral Nemo.

The project page is here. The full repository is here.

Filed under: ai License: CC-BY-SA-4.0.

Written by: @[email protected]