Based on the work the person has done, if they wanted to make something with an exceedingly long battery life, and the battery setup there already seems to be basically a 1S li ion, they could graft in a back side battery holder for two high quality 18650 cells in parallel. It would probably last weeks.
I often ditch my phone and carry my hotspot device for my internet. The only drawback is that for viewing my texts and OTPs i end up putting the sim in my phone or opening up the web interface on my laptop. This seems to work very well as a dumbphone per se.
This probably depends on your particular dongle model. Back in 3G era, some carriers offered USB modems with SMS locked out in the firmware, but you could flash a firmware from a different operator (also install their variant of software, the only difference being the added SMS tab and a different skin) and then you can text and even use USSD! (No calls, though.)
[1]: like this: https://img.joomcdn.net/ea91b4418997d83d802339fad0bbcf878b9d...
---
Aside, but it appears some MSM8916-based dongles run Android UI despite having no display: https://github.com/u0d7i/uz801#screenshots
But I guess you can install a custom messaging app that would sync the messages? (If there’s one that still supports such an old version of Android)