Today, after a long while of having known the wrong configuration for the colours to look alright in and out of tmux along with neovim's
termbuicolors feature, And I did not realise that the solution was right in from of me this whole time.
So, since I am using alacritty as my terminal emulator, it sets the
$TERM variable to
alacritty. Okay, good. Now, the correct config for tmux + alacritty for it to display correct colours with
termguicolors is just as simple as:
set-option -g default-terminal "screen-256color"
set-option -sa terminal-features ',alacritty:RGB'
And that's it! It works wonderfully outside tmux, inside tmux, in neovim, etc...