Chenyo's org-static-blog

Posts tagged "tool":

08 Sep 2024

Build a free Telegram sticker tag bot

1. What happened

When I started to use Telegram, I really missed being able to search stickers by text, like I can in WeChat. Sadly, Telegram only lets you search emojis by text, or find stickers using emojis.

After digging around, I discovered the easiest way to add this cool feature was through Telegram bots. The idea? Store a tag-to-sticker map in the bot, then fetch all matching stickers when given a tag in the bot’s inline mode. There are a few sticker tag bots on Telegram already, but they’re either dead or can’t handle Unicode input like Chinese characters. Plus, I’m not super keen on trusting my data to someone else’s database. Moreover, I might want to use a bot for other personal stuff later.

So, I decided to build my own Telegram bot.

2. What do I need

My goal was to create a bot using only free services, including cloud storage for key-value pairs and a hosting platform to keep the bot running.

I stumbled upon Render from an X recommendation which offers 750 hours per month for free deployment (which equals 31 days), so I deployed my bot there once I got the bot running locally. But then I found out Render’s free tier doesn’t offer permanent storage and shuts down services after 15 minutes of inactivity.

A sticker tag bot without memory isn’t much use to anyone, so I went hunting for free cloud storage. With some help from Claude and Perplexity, I discovered Firebase Realtime database, which offers 1GB storage and 10GB throughput per month on its free tier.

Even with cloud storage, a bot that konks out every 15 minutes just won’t cut it - I need my stickers now! So my next quest was finding a way to keep the bot awake, which led me to UptimeRobot. It’s actually a web monitoring tool, but I can use it to ping the bot regularly, tricking it into staying “active” instead of dozing off.

So, to sum it up, building this sticker tag bot required:

  • a Telegram bot from BotFather,
  • a free Render deployment,
  • a Firebase’s free key-value storage, and
  • an UptimeRobot’s free monitoring service.

However, these services do not work together automatically. Gluing them together required additional effort.

3. How to build a bot

The first step in building any bot is asking BotFather for a new bot and keeping the bot token secure. Telegram offers a helpful tutorial that explains the process using Java. Examples in other languages can be found in their Gitlab repo. In my opinion, the most challenging part here is creating a unique bot username that is still available.

The next step involves working with the Telegram bot API in the chosen programming language. This includes learning how to handle messages effectively. For example, I used tgbotapi(Golang).

3.1. How to build a sticker tag bot

A sticker tag bot needs two main functionalities:

  • Handle direct messages to add new sticker tags.
  • Handle inline queries to search for stickers using a given tag.

To implement the first functionality, I created a simple state machine with two states:

  1. The initial state waits for a sticker input and then moves to the tag state.
  2. The tag state waits for any text input to use as the tag for the previous sticker.

To implement the second functionality, one needs to use the InlineQueryResultCachedSticker method.

For local testing, one can use a lightweight local key-value storage to store and search sticker tags. I used BadgerDB(Golang) for example.

I noticed that the generated file ID for the same sticker is different each time, making it hard to check for duplicates when adding new tags. To address this, I added a /delete method to remove tags when needed.

3.2. How to make the bot private

I couldn’t find an official way to make a bot visible only to me. Suggested by Claude, I predefined a list of authorized users. Then I performed a sanity check before handling any messages.

4. How to deploy the bot on Render

Deploying a service on Render with a free account is challenging due to the lack of shell access, disk access, and non-so-live logs. The process of making everything work at this stage was time-consuming and I even contacted Render’s technical support although they only responded “Ok I will close the ticket” after the issue was self-resolved.

Three main steps are required here:

  1. start an HTTP server with several endpoints at the bot, and
  2. configure the web service and environment variables on Render’s dashboard,
  3. configure the Telegram webhook at the bot.

In step 1, starting an HTTP server at 0.0.0.0:<some-port> is necessary. One should also enable GET methods for the root and a health check endpoint to allow Render to scan the service regularly.

In step 2, one needs to fill in the service configuration and environment variables in different boxes. This includes settings such as port, build command, and health check endpoint. The issue I encountered was Render could not scan any port even if I have triple-checked that everything worked fine locally. In the end, I solved this issue by adding the Golang build tag -tags netgo in the build command. Actually this flag was configured by default, but I initially replaced it with a simpler build command.

In step 3, one needs to configure the webhook with the Bot API and to enable the POST method for the webhook at the HTTP server (this can also be handled by the Bot API). The webhook can be https: //<your project>.onrender.com/<your-token> (or another unique URL). This URL informs Telegram where to send and receive all messages for the bot.

5. How to connect to the Firebase

The Firebase Realtime database stores key-value pairs in the JSON format. Connecting the bot with the database requires the following steps:

  1. Create the app and the database on Firebase’s dashboard. Specifically, one needs to store the following 3 values for interaction:
    • The database URL, which looks like https: //<your-app>-default-rtdb.*.firebasedatabase.app.
    • The credential file, which can be downloaded at Project settings->Service accounts->Firebase Admin SDK (and should also be added to Render).
  2. Import the language-specific Firebase API to configure the database in the bot. For example, I use firebase(Golang).
  3. Update the database rules in Firebase dashboard to only allow authorized writes for specific tags, e.g., one name/path to refer to those key-value pairs.

It’s worth noting that connecting to the database on Render may take some time after a fresh start. During this initialization period, the log may display a 502 Bad Gateway error to the database.

6. How to configure UptimeRobot

Before configuring UptimeRobot, an attempt was made to ping the bot from within itself, but this approach did not function for Render.

Using UptimeRobot to maintain the bot’s active status involves two primary steps:

  1. Enable the HEAD method (the sole method available for a free account) for any endpoint on the HTTP server.
  2. Configure an HTTP(S) monitor for that endpoint, which appears as <you-project>.onrender.com/<endpoint>, and establish the monitoring interval to less than 15 minutes.

7. Conclusion

This post isn’t meant to be a step-by-step guide for building a Telegram bot. It skips some steps and doesn’t include screenshots. But don’t worry, most of the missing bits can be figured out using AI language models these days. The rest really depends on each specific situation. The main point here is to show how to set up a free small web service, even when there’s no single platform that does it all.

When I first wrote this, my bot had been up and running for 10 days. It only had 30 minutes of downtime, which I think happened because UptimeRobot couldn’t reach Render’s IP address during that time.

Right now, the repository is private since I plan to add a second functionality to the bot soon.

Tags: tool telegram
07 Sep 2024

Install Doom Emacs with Lisp native compilation in WSL2

Today I installed Doom Emacs for my husband on his WSL2. Although the entire process was guided by Claude, there were some back-and-forth during the interaction. Therefore I would like to record the full commands I have used here in sequence for any potential reference.

1. Assumptions

This installation guide assumes a fresh installation of WSL2 Ubuntu 22.04 on Windows 11 in 2024 September.

2. Install prerequisite packages

According to the Doom Emacs documentation, the following packages are recommended:

  • Git 2.23+: this is already installed by default.
  • Emacs 29.4 with Lisp native compilation: this is finicky and will be elaborated later.
  • ripgrep 14.0+: the documentation says 11.0+ suffices, but doom doctor still complains the latest version (13.0) installed from apt is not advanced, so we need to install it from its Github released package later.
  • GNU find: also installed already.
  • fd: sudo apt install fd-find suffices.

3. Install Emacs 29.4

3.1. Before building

First, let’s install some build dependencies:

sudo apt update
sudo apt upgrade  # update packages
sudo apt install build-essential libjansson4 libjansson-dev \
    libgnutls28-dev libtree-sitter-dev libsqlite3-dev
sudo apt install texinfo  # to generate documentation

build-essnetial should install necessary tools to build C/C++ programs such as gcc, g++, make, gdb and dpkg. The rest packages install pre-compiled libraries.

Besides these packages, there are two important packages to support List native compilation:

sudo apt install ibgccjit0 libgccjit-11-dev  # 11 is my gcc version

After installing them, make sure to export the path in the current session, otherwise the compiler will not realize it.

export LD_LIBRARY_PATH=/usr/lib/gcc/x86_64-linux-gnu/11:$LD_LIBRARY_PATH

The last thing to do is install a bunch of X and GTK-3 development libraries for Emacs GUI, and another bunch of image processing libraries.

sudo apt install libx11-dev libtiff-dev libgtk-3-dev libncurses-dev
sudo apt install libtiff5-dev libgif-dev libjpeg-dev libpng-dev libxpm-dev

Without the above packages, one may encounter the following error when configuring the Emacs build:

You seem to be running X, but no X development libraries were found. You should install the relevant development files for X and for the toolkit you want, such as Gtk+ or Motif. Also make sure you have development files for image handling, i.e. tiff, gif, jpeg, png and xpm.

3.2. Build Emacs 29.4 with native-comp

At this moment, we can start to download Emacs source code:

wget https://ftp.gnu.org/gnu/emacs/emacs-29.4.tar.xz
tar xvf emacs-29.4.tar.xz
cd emacs-29.4

Then we can configure the build (i.e., generate Makefile) with the following command.

./configure --with-native-compilation --with-x-toolkit=gtk3 --without-pop
  • --with-native-compilation: with this flag the Emacs source code is compiled to native machine code to achieve better performance.
    • Otherwise it is compiled to bytecode and then interpreted by Emacs virtual machine during runtime.
  • --with-x-toolkit=gtk3: this is recommended by Claude.
  • --without-pop: if we are not using Emacs as the email client, we don’t need to bother configure the protocol.

If everything goes well, one should see the following line in the output. If not, make sure libgccjit has been installed and exported.

Does Emacs have native lisp compiler? yes

Now we can finally start compiling the Emacs:

make -j$(nproc)

If some error occurs, we may want to start again, to do this:

sudo apt install autoconf automake
rm -f Makefile
./autogen.sh  # regenerate the configuration file
# then rebuild
make -j$(nproc)

Finally, install Emacs globally:

sudo make install

To confirm the Emacs indeed used the native Lisp compiler, one can evaluate inside the vanilla Emacs with M-: (M is Alt in WSL2):

(native-comp-available-p) ;; should return t

Congratulations! You have now installed the latest and fastest Emacs on WSL2.

4. Install ripgrep

As mentioned in ripgrep documentation, for Debian/Ubuntu users, one should install the latest ripgrep 14.0+ with the following commands.

curl -LO https://github.com/BurntSushi/ripgrep/releases/download/14.1.0/ripgrep_14.1.0-1_amd64.deb  # check the latest version on its documentation
sudo dpkg -i ripgrep_14.1.0-1_amd64.deb  # dpkg has been installed before

Instead, if one installs it with apt, a 13.0+ version is installed and running doom doctor later returns the warning:

The installed ripgrep binary was not built with support for PCRE lookaheads.

5. Install Doom Emacs

Installing Doom Emacs is straightforward, but before that, one should first remove the default Emacs configuration folder:

rm -rf ~/.emacs.d

Then, clone and install Doom Emacs, it could take a while.

git clone --depth 1 https://github.com/doomemacs/doomemacs ~/.config/emacs
~/.config/emacs/bin/doom install

Don’t forget to export ~/.config/emacs/bin to PATH:

echo 'export PATH="$HOME/.config/emacs/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

Now one can run doom doctor to check any missing dependencies, e.g., shellcheck. One common issue is the Nerd font is not installed by default so that some icons are not properly displayed. To fix that, run M-x nerd-icons-install-font inside the Emacs, then update the font cache with:

fc-cache -fv
# fc-list | grep Nerd  # to verify the font is installed

6. Some issues with running Emacs in WSL2

  • The first thing is I cannot reload the configuration with M-x doom/reload as running this command always gives me the following error message so that I need to restart the Emacs every time the configuration is changed.

    %s sync -B -e /bin/bash: line 1: fg: no job control

  • I really dislike the white border that surrounds any application launched by WSL!
Tags: linux emacs tool
24 Jun 2024

Linux use tips

1. i3

1.1. Move specific workspaces between different monitors (ref)

  1. Adjust the monitor relative positions.
  2. Use i3-msg -- move workspace to output right to move the current workspace to the monitor on the right

2. Org

2.1. Format code blocks

  • Use the shortcut < s <TAB> to create a code block.
  • Use C-c ' to enter the code environment to use the language major mode.

3. Emacs

3.1. Prefix argument

  • When the function includes (interactive "P"), it means one can add a prefix argument C-u 1 (or any integer) to trigger some effect.

3.2. Help functions

  • Ctrl-h K get the function name given a shortcut

4. Vim

4.1. Delete a word backwards

  • diw or daw

5. Firefox

5.1. Cache bypass refresh

  • Ctrl+Shift+R
Tags: tool linux i3 arch
Other posts