A "category" (or "module") is now displayed with each log message
together with the source function. The name of the category is derived
from the source file name (see modname() in log.c).
Taisei now also loads a "logfilter" file from the storage directory that
controls which messages are propagated to the console. You can filter by
log level, category, and function. Simple glob pattern matching is
supported. Each line contains a new rule; empty lines and lines starting
with # are ignored. Rules are processed from top to bottom.
Example logfilter file:
# [mask] [category] [function]
# Hide debug messages by default:
-d * *
# Show debug messages from all of the rendering-related modules:
+d renderer/* *
# Also show debug messages from my specific function:
+d * my_function
# Hide info about resource loading/unloading:
-i resource unload_resource
-i resource load_resource_finish
# Suppress everything from a specific module:
-a spam *
Note that only the console output is filtered; the log file still
contains all messages. In addition to that, the log file now also
contains the filename and line number of the original log call with
every message.
The output of __FILE__ previously depended on where the build directory
is and was needlessly verbose. With this change, all filenames are
relative to the src/ directory.
Logging of source file names is now also enabled for release builds
(although this is not used yet).
* Implement asynchronous logging
Put all log messages into a queue. A background thread will format and
dispatch the messages to loggers in a first-in-first-out fashion. This
is mostly useful for windows/wine, or in cases *very* spammy logging.
This adds some malloc overhead, however.
Can be disabled with TAISEI_LOG_ASYNC=0
* log: unfuck memory management
* log: add TAISEI_LOG_ASYNC_FAST_SHUTDOWN env variable
If true, don't wait for the logging queue to process every message
before shutting down.
* strbuf: fix off-by-one error
* log: sync queue before crashing on assertion failure
* docs/ENVIRON: update logging docs
* WIP compressed textures, swizzles, sRGB sampling, ...
* refactor texture type info & fix random bugs
* fix preprocessing of sRGB textures
* handle y-flipped basis textures
* glcommon: better WebGL compat for compressed format detection
* missed WEBGL_compressed_texture_pvrtc
* implement compressed texture xcoding and uploading
* Add basis_universal submodule
* Reorganize texture loader code
Clean up some code
Isolate Basis Universal loader into a separate module
* Add wrapper script for encoding .basis textures
* basisu: honor custom metadata written by the mkbasis.py script
* mkbasis.py: add --incredibly-slow and --dry-run
* Move pixmap code from util/ to pixmap/
* Add an on-disk transcode cache for basis textures to speed up loads
* Compress texture cache with zlib
* Use readable format names for basisu cache filenames
* basisu: mip bias test code
* basisu: small caching cleanup
* add TAISEI_BASISU_MIP_BIAS env variable
* Improve OpenGL format matching heuristics
* Document considerations for compressed format priority
* Remove dead code
* Enable two forgotten formats, BC3_RGBA and ATC_RGBA
Also prefer BC7 over BC1/BC3
* Recognize GL_ANGLE_compressed_texture_etc for ETC2 textures
* Default depth buffers to 24-bit; remove ANGLE hack
* Fix glcommon_check_extension for GLES2/legacy gl
* Add renderer feature bit for texture swizzle masks
* glcommon: Fixup internal formats for GLES2
Sized internal formats are not allowed in GLES2
* Fix emscripten compile errors
* Update basis_universal
* remove more dead code
* revert irrelevant stage4 change
* shut up UBSan
* basisu: shut up some debug spam
* Add normalmap sampling helper to util.glslh
* basisu: add a gray-alpha mode
* mkbasis.py: Abort if image dimansions aren't multiples of 4
* Add basic Basis Universal encoding documentation (WIP)
* doc/basisu: Add paragraph about modes; minor tweaks
* basisu: workarounds for GL texture size requirements
* gles20: fix uncompressed sRGB formats
* Partial workaround for missing swizzles in gles2 and webgl
* remove invalid assertion
* New renderer API to expose glDrawBuffers-like functionality
* stagedraw: disable all color outputs for copy_depth pass
required for WebGL compatibility
* support GL_ANGLE_request_extension
* emscripten: include *.basis in gfx package
Also fix a potential problem when more than one .pkgdir is used to
construct emscripten packages
* Don't rely on emscripten runtime to enable webgl extensions
- RESF_UNSAFE is removed.
- Resources that don't have to be finalized on the main thread can load
completely asynchronously.
- A thread waiting for a concurrent task to complete can start executing
that task itself if it hasn't started yet.
- Refactor the resource loading interface, add support for load-time
dependencies.
- Main-thread finalization of asynchronously loaded resources is now
spread out across multiple frames to mitigate frametime spikes.
- Remove some archaisms from the resource management code.
- Fix potential hashtable synchronization issue.
- Fix some deadlock edge cases.
- Don't spawn more worker threads than there are CPU cores (degrades
performance).
- Add TAISEI_AGGRESSIVE_PRELOAD env variable to attempt to aggressively
discover and preload every possible resource.
- Make r_texture_fill{,_region} expect optimal pixmaps, so that it's
never forced to convert them on the main thread. The optimal format may
be queried with the new r_texture_optimal_pixmap_format_for_type API.
These functions will also no longer needlessly copy the entire image
into a staging buffer - previously they did this even if no conversion
was needed.
- Other random changes to facilitate the stuff above.
The overall effect is somewhat faster load times.
Of course it's still all terrible and full of lock contention because I
suck at concurrent programming, but it's not worse than it was.
Probably.
* Major refactoring of the main loop(s) and control flow (WIP)
run_at_fps() is gone 🦀
Instead of nested blocking event loops, there is now an eventloop API
that manages an explicit stack of scenes. This makes Taisei a lot more
portable to async environments where spinning a loop forever without
yielding control simply is not an option, and that is the entire point
of this change.
A prime example of such an environment is the Web (via emscripten).
Taisei was able to run there through a terrible hack: inserting
emscripten_sleep calls into the loop, which would yield to the browser.
This has several major drawbacks: first of all, every function that
could possibly call emscripten_sleep must be compiled into a special
kind of bytecode, which then has to be interpreted at runtime, *much*
slower than JITed WebAssembly. And that includes *everything* down the
call stack, too! For more information, see
https://emscripten.org/docs/porting/emterpreter.html
Even though that method worked well enough for experimenting, despite
suboptimal performance, there is another obvious drawback:
emscripten_sleep is implemented via setTimeout(), which can be very
imprecise and is generally not reliable for fluid animation. Browsers
actually have an API specifically for that use case:
window.requestAnimationFrame(), but Taisei's original blocking control
flow style is simply not compatible with it. Emscripten exposes this API
with its emscripten_set_main_loop(), which the eventloop backend now
uses on that platform.
Unfortunately, C is still C, with no fancy closures or coroutines.
With blocking calls into menu/scene loops gone, the control flow is
reimplemented via so-called (pun intended) "call chains". That is
basically an euphemism for callback hell. With manual memory management
and zero type-safety. Not that the menu system wasn't shitty enough
already. I'll just keep telling myself that this is all temporary and
will be replaced with scripts in v1.4.
* improve build system for emscripten + various fixes
* squish menu bugs
* improve emscripten event loop; disable EMULATE_FUNCTION_POINTER_CASTS
Note that stock freetype does not work without
EMULATE_FUNCTION_POINTER_CASTS; use a patched version from the
"emscripten" branch here:
https://github.com/taisei-project/freetype2/tree/emscripten
* Enable -Wcast-function-type
Calling functions through incompatible pointers is nasal demons and
doesn't work in WASM.
* webgl: workaround a crash on some browsers
* emscripten improvements:
* Persist state (config, progress, replays, ...) in local IndexDB
* Simpler HTML shell (temporary)
* Enable more optimizations
* fix build if validate_glsl=false
* emscripten: improve asset packaging, with local cache
Note that even though there are rules to build audio bundles, audio
does *not* work yet. It looks like SDL2_mixer can not work without
threads, which is a problem. Yet another reason to write an OpenAL
backend - emscripten supports that natively.
* emscripten: customize the html shell
* emscripten: force "show log" checkbox unchecked initially
* emscripten: remove quit shortcut from main menu (since there's no quit)
* emscripten: log area fixes
* emscripten/webgl: workaround for fullscreen viewport issue
* emscripten: implement frameskip
* emscripter: improve framerate limiter
* align List to at least 8 bytes (shut up warnings)
* fix non-emscripten builds
* improve fullscreen handling, mainly for emscripten
* Workaround to make audio work in chromium
emscripten-core/emscripten#6511
* emscripten: better vsync handling; enable vsync & disable fxaa by default
* Add a general purpose multi-threaded task manager (worker pool) for background tasks
Reimplemented screenshots off-loading using the new task manager.
* Largerly rewrite resource loading internals
They use the new task manager API now and should be generally more
robust.
* Made the game playable without threads again
* wait for resource async load task instead of intermediate state change
* remove dead code
* taskmgr: if creating a worker thread fails, try to make sure the others terminate
I would've preferred to just go with 4-spaces for indent and no tabs,
but lao is a bit conservative about it. :^)
Still, this is a ton better than mixing different styles all over the
place, especially within the same file.
Enabled only if the environment has the execinfo.h interface.
By default dumps the backtrace only on log_fatal, use.
TAISEI_LOGLVLS_BACKTRACE to customize that behaviour, e.g.:
TAISEI_LOGLVLS_BACKTRACE=+w taisei
to also enable it for warnings.
In release builds, everything is logged to the log file by default, and
only warnings and errors go to the console (stderr). In debug builds,
all output goes both to the console (stdout/stderr) and the log file.
You can override this behaviour with environment variables. Some
examples:
Don't log anything to the console at all:
TAISEI_LOGLVLS_CONSOLE=-a
Suppress info and debug output in the log file:
TAISEI_LOGLVLS_FILE=-di
More verbose console output in release builds:
TAISEI_LOGLVLS_CONSOLE=+i
Override defaults and enable all enable the wanted loglevels explicitly:
TAISEI_LOGLVLS_CONSOLE=-a+diwe
Redirect debug output to stderr:
TAISEI_LOGLVLS_STDOUT=-d
TAISEI_LOGLVLS_STDERR=+d
The goal of this change is mainly to clean up Taisei's codebase and
improve its console output. I've been frustrated by files littered with
inconsistent printf/fprintf/warnx/errx calls for a long time, and now I
actually did something about it.
All the above functions are now considered deprecated and result in a
compile-time warning when used. Instead, the following macros should be
used:
log_debug(format, ...)
log_info(format, ...)
log_warn(format, ...)
log_err(format, ...)
As you can see, all of them have the same printf-like interface. But
they have different functionality and purpose:
log_debug is intended for very verbose and specific information. It
does nothing in release builds, much like assert(), so don't use
expressions with side-effects in its arguments.
log_info is for various status updates that are expected during
normal operation of the program.
log_warn is for non-critical failures or other things that may be
worth investigating, but don't inherently render the program
non-functional.
log_err is for when the only choice is to give up. Like errx, it
also terminates the program. Unlike errx, it actually calls abort(),
which means the cleanup functions are not ran -- but on the other
hand, you get a debuggable backtrace. However, if you're trying to
catch programming errors, consider using assert() instead.
All of them produce output that contains a timestamp, the log level
identifier, the calling function's name, and the formatted message.
The newline at the end of the format string is not required -- no, it is
actually *prohibited*. The logging system will take care of the line
breaks by itself, don't litter the code with that shit.
Internally, the logging system is based on the SDL_RWops abstraction,
and may have multiple, configurable destinations. This makes it easily
extensible. Currently, log_debug and log_info are set to write to
stdout, log_warn and log_err to stderr, and all of them also to the file
log.txt in the Taisei config directory.
Consequently, the nasty freopen hacks we used to make Taisei write to
log files on Windows are no longer needed -- which is a very good thing,
considering they probably would break if the configdir path contains
UTF-8 characters. SDL_RWFromFile does not suffer this limitation.
As an added bonus, it's also thread-safe.
Note about printf and fprintf: in very few cases, the logging system is
not a good substitute for these functions. That is, when you care about
writing exactly to stdout/stderr and about exactly how the output looks.
However, I insist on keeping the deprecation warnings on them to not
tempt anyone to use them for logging/debugging out of habit and/or
laziness.
For this reason, I've added a tsfprintf function to util.c. It is
functionally identical to fprintf, except it returns void. Yes, the name
is deliberately ugly. Avoid using it if possible, but if you must, only
use it to write to stdout or stderr. Do not write to actual files with
it, use SDL_RWops.
