feat: publishing infernet-container-starter v0.2.0

projects/tgi-llm/container/README.md

@@ -0,0 +1,88 @@
+# TGI LLM
+
+In this example, we're running an infernet node along with a TGI service.
+
+## Deploying TGI Service
+
+If you have your own TGI service running, feel free to skip this part. Otherwise,
+you can deploy the TGI service using the following command.
+
+Make sure you have a machine with proper GPU support. Clone this repository &
+run the following command:
+
+```bash
+make run-service project=tgi-llm service=tgi
+```
+
+## Deploying Infernet Node Locally
+
+Running an infernet node involves a simple configuration step & running step.
+
+### Configuration
+
+Copy our [sample config file](./config.sample.json) into a new file
+called `config.json`.
+
+```bash
+cp config.sample.json config.json
+```
+
+Then provide the `"env"` field of the `"containers"` section of the file to point to the
+TGI Service you just deployed.
+
+```json
+{
+  // etc.
+  "containers": [
+    {
+      "id": "tgi-llm",
+      "image": "ritualnetwork/llm_inference_service:latest",
+      "external": true,
+      "port": "3000",
+      "allowed_delegate_addresses": [],
+      "allowed_addresses": [],
+      "allowed_ips": [],
+      "command": "--bind=0.0.0.0:3000 --workers=2",
+      "env": {
+        "TGI_SERVICE_URL": "http://{your-service-ip}:{your-service-port}" // <- Change this to the TGI service you deployed
+      }
+    }
+  ]
+}
+```
+
+### Running the Infernet Node Locally
+
+With that out of the way, you can now run the infernet node using the following command
+at the top-level directory of this repo:
+
+```
+make deploy-container project=tgi-llm
+```
+
+## Testing the Infernet Node
+
+You can test the infernet node by posting a job in the node's REST api.
+
+```bash
+curl -X POST "http://127.0.0.1:4000/api/jobs" \
+   -H "Content-Type: application/json" \
+   -d '{"containers":["tgi-llm"], "data": {"prompt": "can shrimp actually fry rice?"}}'
+```
+
+You can expect a response similar to the following:
+
+```json
+{
+  "id": "f026c7c2-7027-4c2d-b662-2b48c9433a12"
+}
+```
+
+You can then check the status of the job using the following command:
+
+```bash
+curl -X GET http://127.0.0.1:4000/api/jobs\?id\=f026c7c2-7027-4c2d-b662-2b48c9433a12
+[{"id":"f026c7c2-7027-4c2d-b662-2b48c9433a12","result":{"container":"tgi-llm","output":{"output":"\n\nI\u2019m not sure if this is a real question or not, but I\u2019m"}},"status":"success"}]
+```
+
+Congratulations! You've successfully ran an infernet node with a TGI service.