Looked at the source code to:
batocera.linux/package/batocera/core/batocera-configgen/configgen/configgen/generators/dosbox/dosboxGenerator.py
Looks like you can create a dosbox.cfg in the game dir and it will use that instead of the system default dosbox.conf. Note, it looks like this only works for regular dosbox, and not the libretro dosbox. Perhaps the _info.txt file should be updated mentioning that this file is supported.
I'd love it if libretro dosbox did the same thing, but it appears not to currently. I cannot even figure out how to get libretro dosbox to auto launch into a game since it does not even call the dosbox.bat file.
Also, thx Dragu, I noticed your reply after I figured this out on my own.