diff --git a/README.md b/README.md index 5060756a..af6d24fe 100644 --- a/README.md +++ b/README.md @@ -5,75 +5,14 @@ PluralKit is a Discord bot meant for plural communities. It has features like me PluralKit has a Discord server for support, feedback, and discussion: https://discord.gg/PczBt78 -# Requirements -Running the bot requires [.NET 5](https://dotnet.microsoft.com/download), a PostgreSQL database and a Redis database. It should function on any system where the prerequisites are set up (including Windows). - -Optionally, it can integrate with [Sentry](https://sentry.io/welcome/) for error reporting and [InfluxDB](https://www.influxdata.com/products/influxdb-overview/) for aggregate statistics. - -# Configuration -Configuring the bot is done through a JSON configuration file. An example of the configuration format can be seen in [`pluralkit.conf.example`](https://github.com/PluralKit/PluralKit/blob/master/pluralkit.conf.example). -The configuration file needs to be placed in the bot's working directory (usually the repository root) and must be called `pluralkit.conf`. - -The configuration file is in JSON format (albeit with a `.conf` extension). The following keys are available (using `.` to indicate a nested object level), bolded key names are required: -* **`PluralKit.Bot.Token`**: the Discord bot token to connect with -* **`PluralKit.Database`**: the URI of the database to connect to (in [ADO.NET Npgsql format](https://www.connectionstrings.com/npgsql/)) -* **`PluralKit.RedisAddr`**: the `host:port` of a Redis database to connect to -* `PluralKit.Bot.Prefixes`: an array of command prefixes to use (default `["pk;", "pk!"]`). -* **`PluralKit.Bot.ClientId`**: the ID of the bot's user account, used for calculating the bot's own permissions and for the link in `pk;invite`. -* `PluralKit.SentryUrl` *(optional)*: the [Sentry](https://sentry.io/welcome/) client key/DSN to report runtime errors to. If absent, disables Sentry integration. -* `PluralKit.InfluxUrl` *(optional)*: the URL to an [InfluxDB](https://www.influxdata.com/products/influxdb-overview/) server to report aggregate statistics to. An example of these stats can be seen on [the public stats page](https://stats.pluralkit.me). -* `PluralKit.InfluxDb` *(optional)*: the name of an InfluxDB database to report statistics to. If either this field or `PluralKit.InfluxUrl` are absent, InfluxDB reporting will be disabled. -* `PluralKit.LogDir` *(optional)*: the directory to save information and error logs to. If left blank, will default to `logs/` in the current working directory. - -The bot can also take configuration from environment variables, which will override the values read from the file. Here, use `:` (colon) or `__` (double underscore) as a level separator (eg. `export PluralKit__Bot__Token=foobar123`) as per [ASP.NET config](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-3.1#environment-variables). - # Running +See [RUNNING.md](./RUNNING.md). -## Docker -The easiest way to get the bot running is with Docker. The repository contains a `docker-compose.yml` file ready to use. - -* Clone this repository: `git clone https://github.com/PluralKit/PluralKit` -* Create a `pluralkit.conf` file in the same directory as `docker-compose.yml` containing at least `PluralKit.Bot.Token` and `PluralKit.Bot.ClientId` fields - * (`PluralKit.Database` is overridden in `docker-compose.yml` to point to the Postgres container) -* Build the bot: `docker-compose build` -* Run the bot: `docker-compose up` - -In other words: -``` -$ git clone https://github.com/PluralKit/PluralKit -$ cd PluralKit -$ cp pluralkit.conf.example pluralkit.conf -$ nano pluralkit.conf # (or vim, or whatever) -$ docker-compose up -d -``` - -## Manually -* Install the .NET 6 SDK (see https://dotnet.microsoft.com/download) -* Clone this repository: `git clone https://github.com/PluralKit/PluralKit` -* Create and fill in a `pluralkit.conf` file in the same directory as `docker-compose.yml` -* Run the bot: `dotnet run --project PluralKit.Bot` - * Alternatively, `dotnet build -c Release -o build/`, then `dotnet build/PluralKit.Bot.dll` - -(tip: use `scripts/run-test-db.sh` to run a temporary PostgreSQL database on your local system. Requires Docker.) - -## Scheduled Tasks worker - -There is a scheduled tasks worker that needs to be ran separately from the bot. This handles cleaning up the database, and updating statistics (system/member/etc counts, shown in the `pk;stats` embed). - -Note: This worker is *not required*, and the bot will function correctly without it. - -If you are running the bot via docker-compose, this is set up automatically. - -If you run the bot manually you can run the worker as such: -* `dotnet run --project PluralKit.ScheduledTasks` -* or if you used `dotnet build` rather than `dotnet run` to run the bot: `dotnet build/PluralKit.ScheduledTasks.dll` - -# Upgrading database from legacy version -If you have an instance of the Python version of the bot (from the `legacy` branch), you may need to take extra database migration steps. -For more information, see [LEGACYMIGRATE.md](./LEGACYMIGRATE.md). +# Development +See [the dev-docs/ directory](./dev-docs/README.md) # User documentation See [the docs/ directory](./docs/README.md) # License -This project is under the GNU Affero General Public License, Version 3. It is available at the following link: https://www.gnu.org/licenses/agpl-3.0.en.html \ No newline at end of file +This project is under the GNU Affero General Public License, Version 3. It is available at the following link: https://www.gnu.org/licenses/agpl-3.0.en.html diff --git a/BRANCHRENAME.md b/dev-docs/BRANCHRENAME.md similarity index 100% rename from BRANCHRENAME.md rename to dev-docs/BRANCHRENAME.md diff --git a/LEGACYMIGRATE.md b/dev-docs/LEGACYMIGRATE.md similarity index 100% rename from LEGACYMIGRATE.md rename to dev-docs/LEGACYMIGRATE.md diff --git a/dev-docs/README.md b/dev-docs/README.md new file mode 100644 index 00000000..e712685f --- /dev/null +++ b/dev-docs/README.md @@ -0,0 +1,21 @@ +# PluralKit development documentation + +Most of PluralKit's code is written in C#, and is split into 3 projects: `PluralKit.Core` (supporting libraries), `PluralKit.Bot` (the Discord bot), and `PluralKit.API` (ASP.NET webserver with controllers for most API endpoints). + +There is an ongoing effort to port this code to Rust, and we have a handful of crates already. Currently, the main Rust services are `gateway` (connects to Discord to receive events), `api` (handles authentication and rate-limiting for the public API, as well as a couple of private endpoints), and `scheduled_tasks` (background cron job runner, for statistics and miscellaneous cleanup). + +At the very least, `PluralKit.Bot` and `gateway` are required for the bot to run. While code still exists to connect to the Discord gateway directly from the C# bot, this is no longer a supported configuration and may break in the future. + +Service-specific documentation can be found for the C# services in [dotnet.md](./dotnet.md), and for the Rust services in [rust.md](./rust.md) (todo; there is a temporary mostly-accurate document at [RUNNING.md](./RUNNING.md)). + +## Building/running + +PluralKit uses a PostgreSQL database and a Redis database to store data. User-provided data is stored in Postgres; Redis is used for internal state and transient data such as the command execution cache. It's generally easy to run these in Docker or with the Nix `process-compose`, but any install method should work. + +The production instance of PluralKit uses Docker images built in CI. These take a long time to rebuild and aren't good for development (they're production builds, so it's not possible to hook up a debugger). Instead, it's preferable to install build dependencies locally. This is easy with the provided Nix flake: run `nix develop .#bot` to drop into a shell with all the C# build dependencies available, and `nix develop .#services` to get a shell with the Rust build dependencies. It's also okay to manually install build dependencies if you prefer. + +PluralKit services are configured with environment variables; see service-specific documentation for details. Generally, the configuration from the self-host `docker-compose.yml` should get you started. + +## Upgrading database from legacy version +If you have an instance of the Python version of the bot (from the `legacy` branch), you may need to take extra database migration steps. +For more information, see [LEGACYMIGRATE.md](./LEGACYMIGRATE.md). diff --git a/dev-docs/RUNNING.md b/dev-docs/RUNNING.md new file mode 100644 index 00000000..9dc5de7e --- /dev/null +++ b/dev-docs/RUNNING.md @@ -0,0 +1,69 @@ +# Technical Overview: + +PluralKit is composed of several different parts, some of which optional and not needed for the bot to function in a testing environment. +##### Required: +- Bot (*PluralKit.Bot*) +- Gateway +- PostgreSQL Database +- Redis Database +##### Optional: +- API (*PluralKit.API*) +- Scheduled Tasks (*scheduled_tasks*) + +*Optionally, it can also integrate with Sentry for error reporting, and InfluxDB for aggregate statistics. In production, we use [VictoriaMetrics](https://victoriametrics.com/) InfluxDB competible endpoint, to query metrics in a Prometheus-compatible format. + +The bot and API are built using .NET 8, with other optional components used for scaling (ex. scheduled_tasks) built using Rust. +# Building + Running + +**The below procedures are intended for development and testing use only! + +Newer versions of the bot have moved to using [Nix](https://nixos.org/) for simplified builds. A docker compose file is also provided, but not recommended as it is not actively maintained. + +See [Configuration](./CONFIGURATION.md) for full details on configuration as only the basics will be covered here. +Configuration is done through a JSON configuration file `pluralkit.conf` placed in the bot's working directory. An example of the configuration format can be seen in [pluralkit.conf.example](pluralkit.conf.example). +The minimum configuration needed for the bot to function must include the following: +- **`PluralKit.Bot.Token`**: the Discord bot token to connect with +- **`PluralKit.Bot.ClientId`**: the ID of the bot's user account, used for calculating the bot's own permissions and for the link in `pk;invite` +- **`PluralKit.Database`**: the URI of the PostgreSQL database to connect to (in [ADO.NET Npgsql format](https://www.connectionstrings.com/npgsql/)) +- **`PluralKit.RedisAddr`**: the `host:port` of the Redis database to connect to + +**When running using Docker, you do not need to specify the Postgres or Redis URLs as these will be overwritten by environment variables in the compose file.** + +**When using Nix, the Database URI Username, Password, and Database fields must match what the database was setup with in the `flake.nix` file!** + +The bot can also take configuration from environment variables, which will override the values read from the file. Here, use `:` (colon) or `__` (double underscore) as a level separator (eg. `export PluralKit__Bot__Token=foobar123`) as per [ASP.NET config](https://docs.microsoft.com/en-us/aspnet/core/fundamentals/configuration/?view=aspnetcore-3.1#environment-variables). +## Nix (recommended) +The bot, databases, and services are available to run all in one as a Nix flake (using process-compose-flake). +As of the writing of this document, there are a few caveats with the current flake file. +- The database URI in the config must match the username, password, and database specified during database creation (currently `postgres`, `postgres`, and `pluralkit` respectively). +- Not all services are added to the flake file yet. + +**Additionally, as of the writing of this document, the `pluralkit-gateway` service reads environment variables to get the bot token and database URLs, so you also need to create a `.env` file in the `PluralKit` directory with the following variables:** +``` +pluralkit__db__db_password="postgres" +pluralkit__db__data_db_uri="postgresql://postgres@localhost:5432/pluralkit" +pluralkit__db__data_redis_addr="redis://localhost:6379" +pluralkit__discord__bot_token="BOT_TOKEN_GOES_HERE" +pluralkit__discord__client_id="BOT_CLIENT_ID_GOES_HERE" +pluralkit__discord__client_secret=1 +pluralkit__discord__max_concurrency=1 +``` +**(This should match the username/password/database specified in the flake file and the configuration file)** + +*(assuming you already have Git installed, if not, you can start a shell with git by running `nix-shell -p git`)* +1. Clone the repository: `git clone https://github.com/PluralKit/PluralKit` +2. Create a `pluralkit.conf` configuration file in the `PluralKit` directory + - Again, the DB URI parameters must match what's in the `flake.nix` file +3. Create a `.env` configuration file in the `PluralKit` directory *(see above)* +4. Build and run: `nix run .#dev` + - This will download the dependencies, build, and run PluralKit + - If Nix is not setup to allow flakes, you may need to add `--extra-experimental-features nix-command --extra-experimental-features flakes` to the command + - If the `pluralkit-bot` process fails to run, you can restart it by selecting it and pressing `Ctrl-R` +``` +[nix-shell:~]$ git clone https://github.com/PluralKit/PluralKit +[nix-shell:~]$ cd PluralKit +[nix-shell:~/PluralKit]$ cp pluralkit.conf.example pluralkit.conf +[nix-shell:~/PluralKit]$ nano pluralkit.conf +[nix-shell:~/PluralKit]$ nano .env +[nix-shell:~/PluralKit]$ nix run .#dev +``` diff --git a/pluralkit.conf.example b/pluralkit.conf.example index 44ce1ef0..a705265b 100644 --- a/pluralkit.conf.example +++ b/pluralkit.conf.example @@ -4,7 +4,7 @@ "Token": "BOT_TOKEN_GOES_HERE", "ClientId": 466707357099884544, }, - "Database": "Host=localhost;Port=5432;Username=myusername;Password=mypassword;Database=mydatabasename", - "RedisAddr": "host:port" + "Database": "Host=localhost;Port=5432;Username=postgres;Password=postgres;Database=pluralkit", + "RedisAddr": "127.0.0.1:6379" } } \ No newline at end of file