mirror of
https://github.com/NovaOSS/nova-api.git
synced 2024-11-25 16:23:57 +01:00
Documented almost everything
This commit is contained in:
parent
169f5469a9
commit
eb6768cae5
137
README.md
137
README.md
|
@ -74,6 +74,20 @@ This one's code can be found in the following repository: [github.com/novaoss/no
|
|||
- `screen` (for production)
|
||||
- Cloudflare (for security, anti-DDoS, etc.) - we fully support Cloudflare
|
||||
|
||||
## Staging System
|
||||
This repository has an integrated staging system. It's a simple system that allows you to test the API server before deploying it to production.
|
||||
|
||||
You should definitely set up two databases on MongoDB: `nova-core` and `nova-test`. Please note that `nova-core` is always used for `providerkeys`.
|
||||
|
||||
Put your production `.env` file in `env/.prod.env`. Your test `.env` file should be in `.env`.
|
||||
|
||||
Running `PUSH_TO_PRODUCTION.sh` will:
|
||||
- kill port `2333` (production)
|
||||
- remove all contents of the production directory, set to `/home/nova-prod/` (feel free to change it)
|
||||
- then copy the test directory (generally *this* directory) to the production directory
|
||||
- copy the `.env` file from `env/.prod.env` to `.env`
|
||||
- use `screen` to run the production server on port `2333`
|
||||
|
||||
## Install
|
||||
Assuming you have a new version of Python 3.9+ and pip installed:
|
||||
```py
|
||||
|
@ -108,14 +122,13 @@ pip install alt-profanity-check
|
|||
pip install git+https://github.com/dimitrismistriotis/alt-profanity-check.git
|
||||
```
|
||||
|
||||
|
||||
## `.env` configuration
|
||||
Create a `.env` file, make sure not to reveal any of its contents to anyone, and fill in the required values in the format `KEY=VALUE`. Otherwise, the code won't run.
|
||||
|
||||
### Database
|
||||
Set up a MongoDB database and set `MONGO_URI` to the MongoDB database connection URI. Quotation marks are definetly recommended here!
|
||||
|
||||
### Proxy
|
||||
### Proxy (optional)
|
||||
- `PROXY_TYPE` (optional, defaults to `socks.PROXY_TYPE_HTTP`): the type of proxy - can be `http`, `https`, `socks4`, `socks5`, `4` or `5`, etc...
|
||||
- `PROXY_HOST`: the proxy host (host domain or IP address), without port!
|
||||
- `PROXY_PORT` (optional)
|
||||
|
@ -125,7 +138,7 @@ Set up a MongoDB database and set `MONGO_URI` to the MongoDB database connection
|
|||
Want to use a proxy list? See the according section!
|
||||
Keep in mind to set `USE_PROXY_LIST` to `True`! Otherwise, the proxy list won't be used.
|
||||
|
||||
### Proxy Lists
|
||||
### Proxy Lists (optional)
|
||||
To use proxy lists, navigate to `api/secret/proxies/` and create the following files:
|
||||
- `http.txt`
|
||||
- `socks4.txt`
|
||||
|
@ -176,7 +189,99 @@ You can also just add the *beginning* of an API address, like `12.123.` (without
|
|||
### Other
|
||||
`KEYGEN_INFIX` can be almost any string (avoid spaces or special characters) - this string will be put in the middle of every NovaAI API key which is generated. This is useful for identifying the source of the key using e.g. RegEx.
|
||||
|
||||
## Run
|
||||
## Misc
|
||||
`api/cache/models.json` has to be a valid JSON file in the OpenAI format. It is what `/v1/models` always returns. Make sure to update it regularly.
|
||||
|
||||
Example: https://pastebin.com/raw/WuNzTJDr (updated ~aug/2023)
|
||||
|
||||
## Providers
|
||||
This is one of the most essential parts of NovaAI. Providers are the APIs used to access the AI models.
|
||||
The modules are located in `api/providers/` and active providers are specified in `api/providers/__init__.py`.
|
||||
|
||||
You shouldn't use `.env` for provider keys. Instead, use the database as it's more flexible and secure. For a detailed explanation, scroll down a bit to the database section.
|
||||
|
||||
Always return the `provider_auth` dictionary key if you a API key is required. Example:
|
||||
|
||||
```py
|
||||
...
|
||||
return {
|
||||
...
|
||||
'provider_auth': f'exampleprovider>{key}'
|
||||
}
|
||||
...
|
||||
```
|
||||
|
||||
Whereas `exampleprovider` is the provider name used in the database.
|
||||
|
||||
### Check providers
|
||||
List all providers using `python api`. This **won't** start the API server.
|
||||
|
||||
Check if a provider is working using `python api <provider>`, e.g. `python api azure`. This **doesn't** require the API to be running.
|
||||
|
||||
## Core Database
|
||||
|
||||
You need to set up a MongoDB database and set `MONGO_URI` in `.env` to the MongoDB database connection URI. Use quotation marks!
|
||||
|
||||
It's also important to set up the database `nova-core`.
|
||||
|
||||
The following collections are used in the database and will be created automatically if they don't exist.
|
||||
|
||||
### `users`
|
||||
Generally, the `api_key` should be treated as the primary key. However, the `discord` and `github` keys are also unique and can be used as primary keys.
|
||||
- `api_key`: API key [str]
|
||||
- `credits`: credits [int]
|
||||
- `role` (optional): credit multiplier. Check `api/config/config.yml`. [str]
|
||||
- `status`:
|
||||
- `active`: defaults to `true`. May be used in the future so that users can deactivate their accounts.
|
||||
- `ban_reason`: defaults to `""`. If the user is banned, this will be set to the reason.
|
||||
- `auth`:
|
||||
- `discord`: Discord user ID. Use a string, because Discord IDs can be larger than the maximum integer value.
|
||||
- `github`: GitHub user ID. Not used yet.
|
||||
- `level`: Discord (Arcane bot) level. [int]
|
||||
|
||||
### `providerkeys`
|
||||
Used in `api/providers/...` to store API keys of the providers.
|
||||
|
||||
- `provider`: provider name [str], e.g. `azure``
|
||||
- `key`: API key [str], e.g. `sk-...`
|
||||
- `rate_limited_since`: timestamp [int], defaults to `null`. The unix timestamp when the provider was rate limited. If it's `null`, the provider is not rate limited.
|
||||
- `inactive_reason`: defaults to `null`. If the key is disabled or terminated, this will be set to the reason automatically. You can also set it manually. [str]
|
||||
- `source`: just to keep track of where the key came from. [str]
|
||||
|
||||
### `stats`
|
||||
Logs general statistics.
|
||||
Automatically updated by the API server.
|
||||
|
||||
More info is yet to be documented.
|
||||
|
||||
### `logs`
|
||||
Every API request is logged here.
|
||||
Automatically updated by the API server.
|
||||
|
||||
More info is yet to be documented.
|
||||
|
||||
## Finance Database
|
||||
The finance database is used to keep track of the finances.
|
||||
|
||||
**Important:** *always* use the specified `currency` for the field `amount`.
|
||||
|
||||
### `donations`
|
||||
- `currency`: (crypto) currency [str], e.g. EUR, USD, BTC, USDT-TRX, ...
|
||||
- `amount`: amount [float].
|
||||
- `user`: [str], Discord user ID
|
||||
- `proof`: [str], link to proof (e.g. transaction hash)
|
||||
- `timestamp`: [int], unix timestamp
|
||||
|
||||
### `expenses`
|
||||
- `currency`: (crypto) currency [str], e.g. EUR, USD, BTC, USDT-ETH, ...
|
||||
- `amount`: amount [float]. NOT negative!
|
||||
- `proof`: [str], link to proof (e.g. transaction hash)
|
||||
- `type`: [str], type of expense, e.g. `wage`, `payment`, `donation`, ...
|
||||
- `to`: [str], entity the expense was paid to, e.g. `IPRoyal`, `employee/John`
|
||||
- `reason`: [str], reason for the expense
|
||||
- `timestamp`: [int], unix timestamp
|
||||
|
||||
## Run Test Server
|
||||
> **Warning:** read the according section for production usage!
|
||||
|
||||
For developement:
|
||||
|
@ -193,32 +298,18 @@ You can also specify a port, e.g.:
|
|||
python run 1337
|
||||
```
|
||||
|
||||
## Adding a provider
|
||||
To be documented!]
|
||||
|
||||
## Run tests
|
||||
## Tests
|
||||
Make sure the API server is running on the port you specified and run:
|
||||
`python checks`
|
||||
|
||||
## Default Ports
|
||||
It is recommended to use the default ports, because this will make it easier to set other parts of the infrastructure up.
|
||||
|
||||
```yml
|
||||
2332: Developement
|
||||
2333: Production
|
||||
```
|
||||
|
||||
## Production
|
||||
## Run Production Server
|
||||
|
||||
Make sure your server is secure and up to date.
|
||||
Check everything.
|
||||
|
||||
The following command will run the API __without__ a reloader!
|
||||
|
||||
```bash
|
||||
python run prod
|
||||
```
|
||||
|
||||
or
|
||||
|
||||
```bash
|
||||
./screen.sh
|
||||
```
|
||||
Make sure you have read all the according sections and have set up everything correctly.
|
||||
|
|
|
@ -1,2 +0,0 @@
|
|||
# import pruner
|
||||
# pruner.prune()
|
|
@ -1,86 +0,0 @@
|
|||
# import os
|
||||
# import sys
|
||||
|
||||
# project_root = os.path.abspath(os.path.join(os.path.dirname(__file__), '..'))
|
||||
# sys.path.append(project_root)
|
||||
|
||||
# from api.db.users import UserManager
|
||||
|
||||
# manager = UserManager()
|
||||
|
||||
# async def update_credits(settings=None):
|
||||
# users = await manager.get_all_users()
|
||||
|
||||
# if not settings:
|
||||
# await users.update_many({}, {'$inc': {'credits': 2500}})
|
||||
|
||||
# else:
|
||||
# for key, value in settings.items():
|
||||
# await users.update_many(
|
||||
# {'level': key}, {'$inc': {'credits': int(value)}})
|
||||
|
||||
# get_all_users = manager.get_all_users
|
||||
|
||||
|
||||
# ###
|
||||
|
||||
# import os
|
||||
# import time
|
||||
# import aiohttp
|
||||
# import pymongo
|
||||
# import asyncio
|
||||
# import autocredits
|
||||
|
||||
# from settings import roles
|
||||
# from dotenv import load_dotenv
|
||||
|
||||
# load_dotenv()
|
||||
|
||||
# async def main():
|
||||
# await update_roles()
|
||||
# await autocredits.update_credits(roles)
|
||||
|
||||
# async def update_roles():
|
||||
# async with aiohttp.ClientSession() as session:
|
||||
# try:
|
||||
# async with session.get('http://0.0.0.0:3224/user_ids') as response:
|
||||
# discord_users = await response.json()
|
||||
# except aiohttp.ClientError as e:
|
||||
# print(f'Error: {e}')
|
||||
# return
|
||||
|
||||
# level_role_names = [f'lvl{lvl}' for lvl in range(10, 110, 10)]
|
||||
# users_doc = await autocredits.get_all_users()
|
||||
# users = users_doc.find({})
|
||||
# users = await users.to_list(length=None)
|
||||
|
||||
|
||||
# for user in users:
|
||||
# if not 'auth' in user:
|
||||
# continue
|
||||
|
||||
# discord = str(user['auth']['discord'])
|
||||
|
||||
# for user_id, role_names in discord_users.items():
|
||||
# if user_id == discord:
|
||||
# for role in level_role_names:
|
||||
# if role in role_names:
|
||||
# users_doc.update_one(
|
||||
# {'auth.discord': discord},
|
||||
# {'$set': {'level': role}}
|
||||
# )
|
||||
|
||||
# print(f'Updated {discord} to {role}')
|
||||
|
||||
# return users
|
||||
|
||||
# def launch():
|
||||
# asyncio.run(main())
|
||||
|
||||
# with open('rewards/last_update.txt', 'w', encoding='utf8') as f:
|
||||
# f.write(str(time.time()))
|
||||
|
||||
# # ====================================================================================
|
||||
|
||||
# if __name__ == '__main__':
|
||||
# launch()
|
|
@ -11,6 +11,8 @@ async def after_request(
|
|||
is_chat: bool,
|
||||
model: str,
|
||||
) -> None:
|
||||
"""Runs after every request."""
|
||||
|
||||
if user and incoming_request:
|
||||
await logs.log_api_request(user=user, incoming_request=incoming_request, target_url=target_request['url'])
|
||||
|
||||
|
@ -20,7 +22,7 @@ async def after_request(
|
|||
ip_address = await network.get_ip(incoming_request)
|
||||
|
||||
await stats.manager.add_date()
|
||||
await stats.manager.add_ip_address(ip_address)
|
||||
# await stats.manager.add_ip_address(ip_address)
|
||||
await stats.manager.add_path(path)
|
||||
await stats.manager.add_target(target_request['url'])
|
||||
|
||||
|
|
|
@ -1,8 +1,6 @@
|
|||
import os
|
||||
import json
|
||||
import asyncio
|
||||
import aiofiles
|
||||
import aiofiles.os
|
||||
|
||||
from sys import argv
|
||||
from bson import json_util
|
||||
|
@ -14,13 +12,14 @@ load_dotenv()
|
|||
MONGO_URI = os.environ['MONGO_URI']
|
||||
FILE_DIR = os.path.dirname(os.path.realpath(__file__))
|
||||
|
||||
async def main(output_dir: str):
|
||||
async def main(output_dir: str) -> None:
|
||||
await make_backup(output_dir)
|
||||
|
||||
async def make_backup(output_dir: str):
|
||||
async def make_backup(output_dir: str) -> None:
|
||||
"""Makes a backup of all collections in the database."""
|
||||
output_dir = os.path.join(FILE_DIR, '..', 'backups', output_dir)
|
||||
|
||||
await aiofiles.os.makedirs(output_dir, exist_ok=True)
|
||||
os.makedirs(output_dir, exist_ok=True)
|
||||
|
||||
client = AsyncIOMotorClient(MONGO_URI)
|
||||
databases = await client.list_database_names()
|
||||
|
@ -30,22 +29,24 @@ async def make_backup(output_dir: str):
|
|||
if database == 'local':
|
||||
continue
|
||||
|
||||
await aiofiles.os.makedirs(os.path.join(output_dir, database), exist_ok=True)
|
||||
os.makedirs(os.path.join(output_dir, database), exist_ok=True)
|
||||
|
||||
for collection in databases[database]:
|
||||
print(f'Initiated database backup for {database}/{collection}')
|
||||
print(f'Initiated backup for {database}/{collection}')
|
||||
await make_backup_for_collection(database, collection, output_dir)
|
||||
print(f'Finished backup for {database}/{collection}')
|
||||
|
||||
async def make_backup_for_collection(database, collection, output_dir):
|
||||
async def make_backup_for_collection(database, collection, output_dir) -> None:
|
||||
"""Makes a backup of a collection in the database."""
|
||||
path = os.path.join(output_dir, database, f'{collection}.json')
|
||||
|
||||
client = AsyncIOMotorClient(MONGO_URI)
|
||||
collection = client[database][collection]
|
||||
documents = await collection.find({}).to_list(length=None)
|
||||
|
||||
async with aiofiles.open(path, 'w') as f:
|
||||
with open(path, 'w') as f:
|
||||
for chunk in json.JSONEncoder(default=json_util.default).iterencode(documents):
|
||||
await f.write(chunk)
|
||||
f.write(chunk)
|
||||
|
||||
if __name__ == '__main__':
|
||||
if len(argv) < 2 or len(argv) > 2:
|
||||
|
|
|
@ -1,11 +1,10 @@
|
|||
max-credits: 100001
|
||||
max-credits-owner: 694201337
|
||||
start-credits: 1000
|
||||
start-credits: 1000 # Credits given to new users.
|
||||
|
||||
# Credit cost per API request.
|
||||
costs:
|
||||
other: 5
|
||||
other: 5 # Other endpoints
|
||||
|
||||
chat-models:
|
||||
chat-models: # chat completions
|
||||
gpt-4-32k: 200
|
||||
gpt-4: 50
|
||||
gpt-3: 10
|
||||
|
|
|
@ -14,7 +14,6 @@ import hmac
|
|||
import httpx
|
||||
import fastapi
|
||||
import aiofiles
|
||||
import functools
|
||||
|
||||
from dhooks import Webhook, Embed
|
||||
from dotenv import load_dotenv
|
||||
|
@ -163,6 +162,7 @@ async def get_crypto_price(cryptocurrency: str) -> float:
|
|||
if is_old or cryptocurrency not in cache:
|
||||
async with httpx.AsyncClient() as client:
|
||||
response = await client.get(f'https://api.coinbase.com/v2/prices/{cryptocurrency}-USD/spot')
|
||||
response.raise_for_status()
|
||||
usd_price = float(response.json()['data']['amount'])
|
||||
|
||||
cache[cryptocurrency] = usd_price
|
||||
|
|
|
@ -13,7 +13,9 @@ class FinanceManager:
|
|||
async def _get_collection(self, collection_name: str):
|
||||
return self.conn['finances'][collection_name]
|
||||
|
||||
async def get_entire_financial_history(self):
|
||||
async def get_entire_financial_history(self) -> dict:
|
||||
"""Returns the entire financial history of the organization."""
|
||||
|
||||
donations_db = await self._get_collection('donations')
|
||||
expenses_db = await self._get_collection('expenses')
|
||||
|
||||
|
|
|
@ -19,14 +19,9 @@ conn = AsyncIOMotorClient(os.environ['MONGO_URI'])
|
|||
async def _get_collection(collection_name: str):
|
||||
return conn[os.getenv('MONGO_NAME', 'nova-test')][collection_name]
|
||||
|
||||
async def replacer(text: str, dict_: dict) -> str:
|
||||
# This seems to exist for a very specific and dumb purpose :D
|
||||
for k, v in dict_.items():
|
||||
text = text.replace(k, v)
|
||||
return text
|
||||
|
||||
async def log_api_request(user: dict, incoming_request, target_url: str):
|
||||
"""Logs the API Request into the database."""
|
||||
|
||||
db = await _get_collection('logs')
|
||||
payload = {}
|
||||
|
||||
|
|
|
@ -3,9 +3,6 @@ import time
|
|||
import random
|
||||
import asyncio
|
||||
|
||||
import aiofiles
|
||||
import aiofiles.os
|
||||
from aiocache import cached
|
||||
from dotenv import load_dotenv
|
||||
from cachetools import TTLCache
|
||||
from motor.motor_asyncio import AsyncIOMotorClient
|
||||
|
@ -22,6 +19,8 @@ class KeyManager:
|
|||
return self.conn['nova-core'][collection_name]
|
||||
|
||||
async def add_key(self, provider: str, key: str, source: str='?'):
|
||||
"""Adds a key to the database."""
|
||||
|
||||
db = await self._get_collection('providerkeys')
|
||||
await db.insert_one({
|
||||
'provider': provider,
|
||||
|
@ -32,6 +31,8 @@ class KeyManager:
|
|||
})
|
||||
|
||||
async def get_possible_keys(self, provider: str):
|
||||
"""Returns a list of possible keys for a provider."""
|
||||
|
||||
db = await self._get_collection('providerkeys')
|
||||
keys = await db.find({
|
||||
'provider': provider,
|
||||
|
@ -45,6 +46,8 @@ class KeyManager:
|
|||
return keys
|
||||
|
||||
async def get_key(self, provider: str):
|
||||
"""Returns a random key for a provider."""
|
||||
|
||||
keys = await self.get_possible_keys(provider)
|
||||
|
||||
if not keys:
|
||||
|
@ -55,6 +58,8 @@ class KeyManager:
|
|||
return api_key
|
||||
|
||||
async def rate_limit_key(self, provider: str, key: str, duration: int):
|
||||
"""Rate limits a key for a provider."""
|
||||
|
||||
db = await self._get_collection('providerkeys')
|
||||
await db.update_one({'provider': provider, 'key': key}, {
|
||||
'$set': {
|
||||
|
@ -63,6 +68,8 @@ class KeyManager:
|
|||
})
|
||||
|
||||
async def deactivate_key(self, provider: str, key: str, reason: str):
|
||||
"""Deactivates a key for a provider, and gives a reason."""
|
||||
|
||||
db = await self._get_collection('providerkeys')
|
||||
await db.update_one({'provider': provider, 'key': key}, {
|
||||
'$set': {
|
||||
|
@ -71,12 +78,14 @@ class KeyManager:
|
|||
})
|
||||
|
||||
async def import_all(self):
|
||||
"""Imports all keys from the secret/ folder."""
|
||||
|
||||
db = await self._get_collection('providerkeys')
|
||||
num = 0
|
||||
|
||||
for filename in await aiofiles.os.listdir(os.path.join('api', 'secret')):
|
||||
for filename in os.listdir(os.path.join('api', 'secret')):
|
||||
if filename.endswith('.txt'):
|
||||
async with aiofiles.open(os.path.join('api', 'secret', filename)) as f:
|
||||
async open(os.path.join('api', 'secret', filename)) as f:
|
||||
async for line in f:
|
||||
if not line.strip():
|
||||
continue
|
||||
|
|
|
@ -4,7 +4,6 @@ import os
|
|||
import json
|
||||
import yaml
|
||||
import time
|
||||
import orjson
|
||||
import fastapi
|
||||
|
||||
from dotenv import load_dotenv
|
||||
|
|
|
@ -12,12 +12,7 @@ async def get_ip(request) -> str:
|
|||
if request.headers.get('x-forwarded-for'):
|
||||
xff, *_ = request.headers['x-forwarded-for'].split(', ')
|
||||
|
||||
possible_ips = [
|
||||
xff,
|
||||
request.headers.get('cf-connecting-ip'),
|
||||
request.client.host
|
||||
]
|
||||
|
||||
possible_ips = [xff, request.headers.get('cf-connecting-ip'), request.client.host]
|
||||
detected_ip = next((i for i in possible_ips if i), None)
|
||||
|
||||
return detected_ip
|
||||
|
|
|
@ -11,7 +11,7 @@ async def _get_module_name(module) -> str:
|
|||
|
||||
async def balance_chat_request(payload: dict) -> dict:
|
||||
"""
|
||||
### Load balance the chat completion request between chat providers.
|
||||
Load balance the chat completion request between chat providers.
|
||||
Providers are sorted by streaming and models. Target (provider.chat_completion) is returned
|
||||
"""
|
||||
|
||||
|
@ -38,7 +38,7 @@ async def balance_chat_request(payload: dict) -> dict:
|
|||
|
||||
async def balance_organic_request(request: dict) -> dict:
|
||||
"""
|
||||
### Load balance non-chat completion request
|
||||
Load balance non-chat completion request
|
||||
Balances between other "organic" providers which respond in the desired format already.
|
||||
Organic providers are used for non-chat completions, such as moderation and other paths.
|
||||
"""
|
||||
|
|
14
api/main.py
14
api/main.py
|
@ -1,9 +1,9 @@
|
|||
"""FastAPI setup."""
|
||||
|
||||
import os
|
||||
import fastapi
|
||||
import pydantic
|
||||
|
||||
from rich import print
|
||||
from dotenv import load_dotenv
|
||||
|
||||
from bson.objectid import ObjectId
|
||||
|
@ -50,11 +50,17 @@ async def startup_event():
|
|||
# https://stackoverflow.com/a/74529009
|
||||
pydantic.json.ENCODERS_BY_TYPE[ObjectId]=str
|
||||
|
||||
folders = ['backups', 'cache', 'secret', 'secret/proxies']
|
||||
|
||||
for folder in folders:
|
||||
try:
|
||||
os.mkdir(folder)
|
||||
except FileExistsError:
|
||||
pass
|
||||
|
||||
@app.get('/')
|
||||
async def root():
|
||||
"""
|
||||
Returns general information about the API.
|
||||
"""
|
||||
"""Returns general information about the API."""
|
||||
|
||||
return {
|
||||
'hi': 'Welcome to the Nova API!',
|
||||
|
|
|
@ -1,13 +1,10 @@
|
|||
"""This module contains functions for checking if a message violates the moderation policy."""
|
||||
|
||||
import time
|
||||
import difflib
|
||||
import asyncio
|
||||
import aiocache
|
||||
import profanity_check
|
||||
|
||||
from typing import Union
|
||||
from Levenshtein import distance
|
||||
|
||||
cache = aiocache.Cache(aiocache.SimpleMemoryCache)
|
||||
|
||||
|
@ -28,8 +25,7 @@ def input_to_text(inp: Union[str, list]) -> str:
|
|||
return text
|
||||
|
||||
async def is_policy_violated(inp: Union[str, list]) -> bool:
|
||||
"""Checks if the input violates the moderation policy.
|
||||
"""
|
||||
"""Checks if the input violates the moderation policy."""
|
||||
# use aio cache to cache the result
|
||||
inp = input_to_text(inp)
|
||||
|
||||
|
|
|
@ -1,2 +1,2 @@
|
|||
from . import azure, webraft
|
||||
MODULES = [azure, webraft]
|
||||
from . import azure
|
||||
MODULES = [azure]
|
||||
|
|
|
@ -3,7 +3,6 @@ import sys
|
|||
import aiohttp
|
||||
import asyncio
|
||||
import importlib
|
||||
import aiofiles.os
|
||||
|
||||
from rich import print
|
||||
|
||||
|
@ -21,7 +20,7 @@ async def main():
|
|||
except IndexError:
|
||||
print('List of available providers:')
|
||||
|
||||
for file_name in await aiofiles.os.listdir(os.path.dirname(__file__)):
|
||||
for file_name in os.listdir(os.path.dirname(__file__)):
|
||||
if file_name.endswith('.py') and not file_name.startswith('_'):
|
||||
print(file_name.split('.')[0])
|
||||
|
||||
|
|
|
@ -1,10 +1,10 @@
|
|||
from .helpers import utils
|
||||
|
||||
AUTH = True
|
||||
ORGANIC = False
|
||||
STREAMING = True
|
||||
MODERATIONS = False
|
||||
ENDPOINT = 'https://nova-00001.openai.azure.com'
|
||||
AUTH = True # If the provider requires an API key
|
||||
ORGANIC = False # If all OpenAI endpoints are available on the provider. If false, only a chat completions are available.
|
||||
STREAMING = True # If the provider supports streaming completions
|
||||
ENDPOINT = 'https://nova-00001.openai.azure.com' # (Important: read below) The endpoint for the provider.
|
||||
#! IMPORTANT: If this is an ORGANIC provider, this should be the endpoint for the API with anything BEFORE the "/v1".
|
||||
MODELS = [
|
||||
'gpt-3.5-turbo',
|
||||
'gpt-3.5-turbo-16k',
|
||||
|
|
|
@ -3,7 +3,6 @@ from .helpers import utils
|
|||
AUTH = True
|
||||
ORGANIC = True
|
||||
STREAMING = True
|
||||
MODERATIONS = True
|
||||
ENDPOINT = 'https://api.openai.com'
|
||||
MODELS = utils.GPT_3
|
||||
|
||||
|
|
|
@ -3,7 +3,6 @@ from .helpers import utils
|
|||
AUTH = True
|
||||
ORGANIC = False
|
||||
STREAMING = True
|
||||
MODERATIONS = True
|
||||
ENDPOINT = 'https://api.openai.com'
|
||||
MODELS = utils.GPT_4
|
||||
|
||||
|
|
|
@ -3,7 +3,6 @@ from .helpers import utils
|
|||
AUTH = True
|
||||
ORGANIC = False
|
||||
STREAMING = True
|
||||
MODERATIONS = False
|
||||
ENDPOINT = 'https://api.openai.com'
|
||||
MODELS = utils.GPT_4_32K
|
||||
|
||||
|
|
|
@ -3,8 +3,6 @@
|
|||
import os
|
||||
import socket
|
||||
import random
|
||||
import asyncio
|
||||
import aiohttp
|
||||
import aiohttp_socks
|
||||
|
||||
from rich import print
|
||||
|
@ -16,7 +14,7 @@ USE_PROXY_LIST = os.getenv('USE_PROXY_LIST', 'False').lower() == 'true'
|
|||
|
||||
class Proxy:
|
||||
"""
|
||||
### Represents a proxy.
|
||||
Represents a proxy.
|
||||
The type can be either http, https, socks4 or socks5.
|
||||
You can also pass a url, which will be parsed into the other attributes.
|
||||
|
||||
|
@ -69,7 +67,7 @@ class Proxy:
|
|||
@property
|
||||
def connector(self):
|
||||
"""
|
||||
### Returns a proxy connector
|
||||
Returns a proxy connector
|
||||
Returns an aiohttp_socks.ProxyConnector object.
|
||||
This can be used in aiohttp.ClientSession.
|
||||
"""
|
||||
|
@ -115,7 +113,7 @@ class ProxyLists:
|
|||
|
||||
def get_proxy() -> Proxy:
|
||||
"""
|
||||
### Returns a Proxy object
|
||||
Returns a Proxy object
|
||||
The proxy is either from the proxy list or from the environment variables.
|
||||
"""
|
||||
|
||||
|
|
|
@ -45,7 +45,8 @@ async def respond(
|
|||
input_tokens: int=0,
|
||||
incoming_request: starlette.requests.Request=None,
|
||||
):
|
||||
"""Stream the completions request. Sends data in chunks
|
||||
"""
|
||||
Stream the completions request. Sends data in chunks
|
||||
If not streaming, it sends the result in its entirety.
|
||||
"""
|
||||
|
||||
|
|
|
@ -1,82 +0,0 @@
|
|||
import os
|
||||
import json
|
||||
import openai
|
||||
|
||||
from dotenv import load_dotenv
|
||||
|
||||
load_dotenv()
|
||||
|
||||
openai.api_base = 'http://localhost:2332/v1'
|
||||
openai.api_key = os.environ['NOVA_KEY']
|
||||
|
||||
# Example dummy function hard coded to return the same weather
|
||||
# In production, this could be your backend API or an external API
|
||||
def get_current_weather(location, unit='fahrenheit'):
|
||||
"""Get the current weather in a given location"""
|
||||
weather_info = {
|
||||
'location': location,
|
||||
'temperature': '72',
|
||||
'unit': unit,
|
||||
'forecast': ['sunny', 'windy'],
|
||||
}
|
||||
return json.dumps(weather_info)
|
||||
|
||||
def run_conversation():
|
||||
# Step 1: send the conversation and available functions to GPT
|
||||
messages = [{'role': 'user', 'content': 'What\'s the weather like in Boston?'}]
|
||||
functions = [
|
||||
{
|
||||
'name': 'get_current_weather',
|
||||
'description': 'Get the current weather in a given location',
|
||||
'parameters': {
|
||||
'type': 'object',
|
||||
'properties': {
|
||||
'location': {
|
||||
'type': 'string',
|
||||
'description': 'The city and state, e.g. San Francisco, CA',
|
||||
},
|
||||
'unit': {'type': 'string', 'enum': ['celsius', 'fahrenheit']},
|
||||
},
|
||||
'required': ['location'],
|
||||
},
|
||||
}
|
||||
]
|
||||
response = openai.ChatCompletion.create(
|
||||
model='gpt-3.5-turbo-0613',
|
||||
messages=messages,
|
||||
functions=functions,
|
||||
function_call='auto', # auto is default, but we'll be explicit
|
||||
)
|
||||
response_message = response['choices'][0]['message']
|
||||
|
||||
# Step 2: check if GPT wanted to call a function
|
||||
if response_message.get('function_call'):
|
||||
# Step 3: call the function
|
||||
# Note: the JSON response may not always be valid; be sure to handle errors
|
||||
available_functions = {
|
||||
'get_current_weather': get_current_weather,
|
||||
} # only one function in this example, but you can have multiple
|
||||
function_name = response_message['function_call']['name']
|
||||
fuction_to_call = available_functions[function_name]
|
||||
function_args = json.loads(response_message['function_call']['arguments'])
|
||||
function_response = fuction_to_call(
|
||||
location=function_args.get('location'),
|
||||
unit=function_args.get('unit'),
|
||||
)
|
||||
|
||||
# Step 4: send the info on the function call and function response to GPT
|
||||
messages.append(response_message) # extend conversation with assistant's reply
|
||||
messages.append(
|
||||
{
|
||||
'role': 'function',
|
||||
'name': function_name,
|
||||
'content': function_response,
|
||||
}
|
||||
) # extend conversation with function response
|
||||
second_response = openai.ChatCompletion.create(
|
||||
model='gpt-3.5-turbo-0613',
|
||||
messages=messages,
|
||||
) # get a new response from GPT where it can see the function response
|
||||
return second_response
|
||||
|
||||
print(run_conversation())
|
|
@ -1,10 +0,0 @@
|
|||
--- EXPECTED ---
|
||||
|
||||
data: {"id":"custom-chatcmpl-nUSiapqELukaPT7vEnGcXkbvrS1fR","object":"chat.completion.chunk","created":1696716717,"model":"gpt-3.5-turbo-0613","choices":[{"index":0,"delta":{"role":"assistant","content":""},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"custom-chatcmpl-nUSiapqELukaPT7vEnGcXkbvrS1fR","object":"chat.completion.chunk","created":1696716717,"model":"gpt-3.5-turbo-0613","choices":[{"index":0,"delta":{"content":"123"},"finish_reason":null}]}
|
||||
|
||||
data: {"id":"custom-chatcmpl-nUSiapqELukaPT7vEnGcXkbvrS1fR","object":"chat.completion.chunk","created":1696716717,"model":"gpt-3.5-turbo-0613","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
|
||||
|
||||
data: [DONE]
|
||||
|
|
@ -9,6 +9,8 @@ from api.db.users import UserManager
|
|||
manager = UserManager()
|
||||
|
||||
async def update_credits(settings=None):
|
||||
"""Updates the credits of all users."""
|
||||
|
||||
users = await manager.get_all_users()
|
||||
|
||||
if not settings:
|
||||
|
|
|
@ -15,6 +15,8 @@ async def main():
|
|||
await autocredits.update_credits(roles)
|
||||
|
||||
async def update_roles():
|
||||
"""Updates the roles of all users."""
|
||||
|
||||
async with aiohttp.ClientSession() as session:
|
||||
try:
|
||||
async with session.get('http://0.0.0.0:3224/get_roles') as response:
|
||||
|
|
Loading…
Reference in a new issue