OpenRouter is a platform that allows users to access and utilize multiple large language models (LLMs) while paying only for the resources they consume. This unique Pay-As-You-Go model provides an affordable and flexible solution for those who require different LLMs or want to avoid premium LLM limits.
OpenRouter offers its users access to a wide array of premium and open models. However, the platform's design might appear slightly confusing at first, and it can be challenging to determine the exact cost of usage without diving into the wallet settings.
Today we’ll take a look at the platform's pros and cons so you can see if it’s right for you.
Usability and Design
OpenRouter's design is minimalist in nature, focusing on simple functionality; however, new users may have trouble determining where to start, especially if they do not have any experience with LLMs.
There are three areas that lay out the models used: “Browse,” “Rankings,” and “Supported Models” under “Docs.” While these have information about popularity, context window, moderation, and prompt costs, there is little information about what a model is good at or how best to get a model to accomplish what you want. For this you will have to do your own research.
If you are still unsure of which model to pick or just want a quick-start, there is an “Auto” option under “Model” in Chat which will have OpenRouter try to pick the best model for your prompt.
Unlike other platforms, OpenRouter only keeps track of the current conversation, and conversations must be saved or imported manually. I was unable to find out whether this means OpenRouter does or doesn't have access to your chat history, but I would hope they do not store this information behind the scenes.
Wallet Management
One of the potential drawbacks of OpenRouter's Pay-As-You-Go model is that it can be challenging for users to track their spending accurately as they use the service. Under Account > Activity, users may review the date/time, model, app, tokens, costs, latency, and speed of each response, but without the text it can be difficult to match this information to the actual use.
The Wallet itself is a little difficult to find: it's under Account > Settings. Here you can see how many credits you have remaining as well as buy more credits. If you don't want to be bothered with monitoring the wallet, there's an “Auto Top Up” feature which will buy a certain amount of credits every time they fall below a set point.
That said, the Wallet/Credit system can be extremely convenient if you are on a budget or simply wish to make sure you get the most out of your cash. Why spend twenty dollars a month on a single premium model if your use only amounts to five dollars?
LLM Access
OpenRouter offers access to a variety of premium and open LLMs, ensuring that users have a range of options to suit their specific needs and preferences.
There are many, many models to choose from. Here is a list of a few of the models currently featured:
MythoMax 13B
Mistral: Mixtral 8x7B Instruct (Beta)
OpenChat 3.5
Nous: Capybara 7B
Google: Gemini Pro (Preview)
Open AI: GPT-4 Turbo (Preview)
Perplexity: PPLX 70B Online
Anthropic: Claude v2.1
OpenRouter makes it simple to also compare models. Simply choose more than one model for your chat, and each model chosen will respond to your prompt. You can then click on each response to expand it and compare them with each other. Keep in mind, you are still paying for each response.
App Access
One of the most useful aspects of OpenRouter is its use as a middleman between LLMs and apps. These apps act as a UI for accessing the LLMs and help you prompt in specific ways. For example, novelcrafter, an app geared toward fiction writers, keeps track of a novel’s text and notes and feeds those into the selected LLM for you as required. This saves the user from continuous streams of copy/pasting and fiddling with prompts to get the best output.
Conclusion
OpenRouter provides users with access to a range of LLMs through its Pay-As-You-Go pricing model. While the design may be slightly confusing at first and tracking spending can be challenging, the platform offers valuable features, such as the flexibility to experiment with different LLMs and the potential for cost-savings.
For users seeking an affordable and flexible solution for utilizing multiple LLMs without a long-term commitment, OpenRouter is worth considering.
Comments