Very slight streaming improvements

.gitignore

@@ -1,3 +1,8 @@
+last_update.txt
+
+!api/providers/__main__.py
+!api/providers/helpers/utils.py
+
 *.log.json
 /logs
 /log

README.md

@@ -40,90 +40,5 @@ This one's code can be found in the following repository: [github.com/novaoss/no
 - In a different repository and with a different domain because it needs to display codes on the website.
 - Implemented in [NovaCord](https://nova-oss.com/novacord)
 
-## Install
+## Install & self-host
-Assuming you have a new version of Python 3 and pip installed:
+See [setup.md](setup.md)
-```py
-python -m pip install -r requirements.txt
-```
-
-If you still get a `ModuleNotFoundError`s, you can forefully install the dependencies using:
-```py
-python -m pip install pipreqs
-python -m pipreqs.pipreqs --force --mode no-pin
-python -m pip install --upgrade -r requirements.txt
-```
-
-You can also try installing Nova API using `setup.py`:
-```py
-python setup.py
-```
-
-or 
-
-```py
-pip install .
-```
-
-## `.env` configuration
-Create a `.env` file, make sure not to reveal it to anyone, and fill in the required values in the format `KEY=VALUE`. Otherwise, the code won't run.
-
-### Database
-- `API_DB_PATH` the path to the databases, e.g. `/etc/nova/db/.` (this way, the database `users` would be saved in `/etc/nova/db/.users.db`.)
-
-### Proxy
-- `PROXY_TYPE` (optional, defaults to `socks.PROXY_TYPE_HTTP`): the type of proxy - can be `http`, `https`, `socks4`, `socks5`, `4` or `5`, etc... 
-- `PROXY_HOST`: the proxy host (host domain or IP address), without port!
-- `PROXY_PORT` (optional)
-- `PROXY_USER` (optional)
-- `PROXY_PASS` (optional)
-
-### `ACTUAL_IPS` (optional)
-This is a security measure to make sure a proxy, VPN, Tor or any other IP hiding service is used by the host when accessing "Closed"AI's API.
-It is a space separated list of IP addresses that are allowed to access the API.
-You can also just add the *beginning* of an API address, like `12.123.` (without an asterisk!) to allow all IPs starting with `12.123.`.
-> To disable the warning if you don't have this feature enabled, set `ACTUAL_IPS` to `None`.
-
-### `CORE_API_KEY`
-This specifies the **very secret key** for accessing the entire user database etc.
-
-## Run
-> **Warning:** read the according section for production usage!
-
-For developement:
-
-```bash
-python run
-```
-
-This will run the development server on port `2332`.
-
-You can also specify a port, e.g.:
-
-```bash
-python run 1337
-```
-
-## Test if it works
-`python tests`
-
-## Ports
-```yml
-2332: Developement (default)
-2333: Production
-```
-
-## Production
-Make sure your server is secure and up to date.
-Check everything.
-
-The following command will run the API  __without__ a reloader!
-
-```bash
-python run prod
-```
-
-or 
-
-```bash
-./screen.sh
-```

api/streaming.py

@@ -140,9 +140,6 @@ async def stream(
                                 chunk = chunk.replace(os.getenv('MAGIC_WORD', 'novaOSScheckKeyword'), payload['model'])
                                 chunk = chunk.replace(os.getenv('MAGIC_USER_WORD', 'novaOSSuserKeyword'), str(user['_id']))
 
-                                if not chunk.strip():
-                                    send = False
-
                                 if is_chat and '{' in chunk:
                                     data = json.loads(chunk.split('data: ')[1])
                                     chunk = chunk.replace(data['id'], chat_id)
@@ -160,7 +157,7 @@ async def stream(
                                     if data['choices'][0]['delta'] == {'role': 'assistant'}:
                                         send = False
                                 
-                                if send:
+                                if send and chunk.strip():
                                     final_chunk = chunk.strip().replace('data: [DONE]', '') + '\n\n'
                                     yield final_chunk
 

rewards/last_update.txt

@@ -1 +0,0 @@
-1691546405.042006

setup.md

@@ -0,0 +1,99 @@
+
+## Install
+Assuming you have a new version of Python 3.9+ and pip installed:
+```py
+python -m pip install -r requirements.txt
+```
+
+If you still get a `ModuleNotFoundError`s, you can forefully install the dependencies using:
+```py
+python -m pip install pipreqs
+python -m pipreqs.pipreqs --force --mode no-pin
+python -m pip install --upgrade -r requirements.txt
+```
+
+You can also try installing Nova API using `setup.py`:
+```py
+python setup.py
+```
+
+or 
+
+```py
+pip install .
+```
+
+## `.env` configuration
+Create a `.env` file, make sure not to reveal any of its contents to anyone, and fill in the required values in the format `KEY=VALUE`. Otherwise, the code won't run.
+
+### Database
+Set up a MongoDB database and set `MONGO_URI` to the MongoDB database connection URI. Quotation marks are definetly recommended here!
+
+### Proxy
+- `PROXY_TYPE` (optional, defaults to `socks.PROXY_TYPE_HTTP`): the type of proxy - can be `http`, `https`, `socks4`, `socks5`, `4` or `5`, etc... 
+- `PROXY_HOST`: the proxy host (host domain or IP address), without port!
+- `PROXY_PORT` (optional)
+- `PROXY_USER` (optional)
+- `PROXY_PASS` (optional)
+
+### `ACTUAL_IPS` (optional)
+This is a security measure to make sure a proxy, VPN, Tor or any other IP hiding service is used by the host when accessing "Closed"AI's API.
+It is a space separated list of IP addresses that are allowed to access the API.
+You can also just add the *beginning* of an API address, like `12.123.` (without an asterisk!) to allow all IPs starting with `12.123.`.
+> To disable the warning if you don't have this feature enabled, set `ACTUAL_IPS` to `None`.
+
+### Timeout
+`TRANSFER_TIMEOUT` seconds to wait until the program throws an exception for if the request takes too long. We recommend rather long times like `120` for two minutes.
+
+### Core Keys
+`CORE_API_KEY` specifies the **very secret key** for  which need to access the entire user database etc.
+`TEST_NOVA_KEY` is the API key the which is used in tests. It should be one with tons of credits.
+
+## Webhooks
+`DISCORD_WEBHOOK__USER_CREATED` is the Discord webhook URL for when a user is created.
+`DISCORD_WEBHOOK__API_ISSUE` is the Discord webhook URL for when an API issue occurs.
+
+## Other
+`KEYGEN_INFIX` can be almost any string (avoid spaces or special characters) - this string will be put in the middle of every NovaAI API key which is generated. This is useful for identifying the source of the key using e.g. RegEx.
+
+## Run
+> **Warning:** read the according section for production usage!
+
+For developement:
+
+```bash
+python run
+```
+
+This will run the development server on port `2332`.
+
+You can also specify a port, e.g.:
+
+```bash
+python run 1337
+```
+
+## Test if it works
+`python tests`
+
+## Ports
+```yml
+2332: Developement (default)
+2333: Production
+```
+
+## Production
+Make sure your server is secure and up to date.
+Check everything.
+
+The following command will run the API  __without__ a reloader!
+
+```bash
+python run prod
+```
+
+or 
+
+```bash
+./screen.sh
+```