Initial project import

This commit is contained in:
Agila Forge Importer committed 2026-07-13 15:20:32 +00:00
commit 23f2d29e32
11957 files changed
+1754577

No files matched your search

+292
View File
@@ -0,0 +1,292 @@
# dtm - Dynamic Template Manager
A simple template manager integrated into the V project, designed to combine the power of V
templates with Veb, without the need to recompile the application with every change.
> **Status:** `x.templating.dtm` is the legacy version of Dynamic Template
> Manager. New projects should use `x.templating.dtm2`, which provides the
> modern runtime renderer and parsed-template cache.
>
> Existing `dtm` users can keep their current API while migrating progressively:
> the v1 facade now delegates rendering to the DTM2 engine internally, preserving
> source compatibility while removing the old rendered-cache server from the
> main rendering path.
## Quick Start
Using the dynamic template manager (named `dtm` in this readme) is relatively straightforward.
You just need to initialize an instance. Then, call the `expand` function in the code that
manages your template.
Before starting, you can specify a folder, but it is not mandatory to store the generated cache. If
nothing is specified or if there is a problem with the targeted folder (for example, a permission
issue), the DTM will attempt to create a cache folder in the temporary file area of your OS.
Finally, if all this does not prove successful, then the DTM will disable the cache system and
notify the user if the cache system was previously required.
However, at the root directory of your project, you need to create a `templates/` folder. The
absence of these directories will prevent you from using the dtm. You must add your template files
into the folder you previously created. ***Be aware that if the templates are not placed in the
correct directory, the DTM will return an error message indicating that it cannot find the
template!***
The DTM currently only works with 2 types of files:
- html
- raw text
Below you will find 2 usage contexts:
### 1. Minimal static generator example :
```v
import x.templating.dtm
fn main() {
mut dtmi := dtm.initialize()
// No need to add this 'defer' if you have chosen to disable the cache system in the options.
defer {
dtmi.stop_cache_handler()
}
mut tmp_var := map[string]dtm.DtmMultiTypeMap{}
tmp_var['title'] = 'V dtm best title'
tmp_var['non_string_type'] = 7
tmp_var['html_section_#includehtml'] = '<span>will <br> be <br> escaped <br> in <br> text mode</span>'
render := dtmi.expand('test.txt', placeholders: &tmp_var)
println(render)
}
```
#### and its template text format:
```
Title of text: @title
Value in the text: @non_string_type
HTML tags are always escaped in text file: @html_section
```
### 2. Minimal Veb example:
```v
import veb
import x.templating.dtm
import os
pub struct App {
pub mut:
dtmi &dtm.DynamicTemplateManager = unsafe { nil }
}
pub struct Context {
veb.Context
}
fn main() {
cache_folder_path := os.join_path(os.dir(os.executable()), 'vcache_dtm')
mut app := &App{
dtmi: dtm.initialize(def_cache_path: cache_folder_path)
}
// No need to add this 'defer' if you have chosen to disable the cache system in the options.
defer {
app.dtmi.stop_cache_handler()
}
// Here is an example of init configuration:
dtm.initialize(
def_cache_path: cache_folder_path
compress_html: false
active_cache_server: false
max_size_data_in_mem: 100
)
veb.run[App, Context](mut app, 18081)
}
@['/']
pub fn (mut app App) index(mut ctx Context) veb.Result {
mut tmpl_var := map[string]dtm.DtmMultiTypeMap{}
tmpl_var['title'] = 'The true title'
html_content := app.dtmi.expand('index.html', placeholders: &tmpl_var)
return ctx.html(html_content)
}
```
#### and its template html format:
```html
<!doctype html>
<html>
<head>
<title>@title</title>
</head>
<body>
<div id="container">
<h1>@title</h1>
</div>
</body>
</html>
```
You have a ready-to-view demonstration available
[here](https://github.com/vlang/v/tree/master/vlib/veb/tests/dynamic_template_manager_test_server).
## Available Options
There are two types of option possibilities:
- Specific to the initialization
- Defined for each call of expand function
### Specific to the initialization
Three parameters are available:
- `def_cache_path` (**string** value): User can define the path of cache folder.
- `max_size_data_in_mem` (**int** value): Maximum size of data allowed in memory for each cached.
template. The value must be specified in kilobytes (default is: 500KB; limit max is: 500KB).
- `compress_html` (**bool** value): Light **minifier** of the HTML output, to remove all
unnecessary spacing (default is `true`, parameter taken into account only for HTML files).
- `active_cache_server` (**bool** value): Activate or not the template cache system (default
is true, ***_highly recommended to keep it enabled for optimal performance_***).
Regarding the `compress_html` option, it is recommended for performance reasons to disable it
when working directly with raw template generation (i.e., with the cache system disabled).
Use it like this:
```v ignore
initialize(
def_cache_path: 'your/directorie/cache/path'
max_size_data_in_mem: 500
compress_html: true
active_cache_server: true
)
```
### Defined for each call of expand function
- `placeholders` (**&map[string]DtmMultiTypeMap** value): Used to map placeholders within the
template to their corresponding content, facilitating dynamic content insertion, by specifying
values in the placeholders map. Templates can dynamically display content.
- `cache_delay_expiration` (**i64** value): Specifies the cache expiration time for the concerned
page in seconds (default value is `86400` seconds or one day). You can add any value you
want in seconds as long as it remains within the indicated range (see below).
Possibility to use already defined cache delay constants like:
- `cache_delay_expiration_at_min`: five minutes.
- `cache_delay_expiration_at_max`: one year.
- `cache_delay_expiration_by_default`: one day.
For specific cases, you can cancel the generation and use of cache file, even if the cache system
is active:
- `cache_delay_expiration: -1`.
Or set a cache that will never expire:
- `cache_delay_expiration: 0`.
Example:
```v ignore
expand('path/of/template.html',
placeholders: &the_map_var
cache_delay_expiration: -1
)
```
## The PlaceHolders System
### On The User's Side Code:
The placeholder system allows for the insertion of dynamic content into your template. As of the
current state of the module, it accepts the following types like:
```
- string
- i8, i16, int, i64
- u8, u16, u32, u64
- f32, f64
```
Example:
```v ignore
mut plhs := map[string]dtm.DtmMultiTypeMap{}
plhs['placeholder_name_1'] = "title content"
plhs['placeholder_name_2'] = 123456
plhs['placeholder_name_3_#includehtml'] = "<p>allow to include</p><span>certain HTML tags</span>"
expand('path/of/template.html',
placeholders: &plhs
)
```
Pay attention to this particular tag: `_#includehtml`, it enables you to include HTML in the
dynamic content. Without this tag, all characters will be escaped for obvious security reasons. By
using this tag, only certain HTML tags are allowed. Here is the list:
```html
'<div>', '</div>', '<h1>', '</h1>', '<h2>', '</h2>', '<h3>', '</h3>', '<h4>',
'</h4>', '<h5>', '</h5>', '<h6>', '</h6>', '<p>', '</p>', '<br>', '<hr>', '<span>', '</span>',
'<ul>', '</ul>', '<ol>', '</ol>', '<li>', '</li>', '<dl>', '</dl>', '<dt>', '</dt>', '<dd>',
'</dd>', '<menu>', '</menu>', '<table>', '</table>', '<caption>', '</caption>', '<th>', '</th>',
'<tr>', '</tr>', '<td>', '</td>', '<thread>', '</thread>', '<tbody>', '</tbody>', '<tfoot>',
'</tfoot>', '<col>', '</col>', '<colgroup>', '</colgroup>', '<header>', '</header>', '<footer>',
'</footer>', '<main>', '</main>', '<section>', '</section>', '<article>', '</article>', '<aside>',
'</aside>', '<details>', '</details>', '<dialog>', '</dialog>', '<data>', '</data>', '<summary>',
'</summary>'
```
#### Note that with a raw text template, all HTML tag inclusions are escaped.
### On The Template Side:
An example of a template, corresponding to the previous subsection:
```html
<!doctype html>
<html>
<head>
<title>@placeholder_name_1</title>
</head>
<body>
<div id="container">
<h1>@placeholder_name_1</h1>
<p>@placeholder_name_2</p>
@placeholder_name_3
</div>
</body>
</html>
```
You will note that the `_#includehtml` directive is not found in the template with
`@placeholder_name_3`, and this is entirely normal. Directives are specially handled
by the DTM, and including them in the name of your placeholders within the template
will result in the placeholder not being found because it does not match the key name
defined in the map containing the dynamic content.
Like the traditional template system in V, inclusions or placeholders start with the `@`
character. The traditional inclusion system is still perfectly usable, such as:
```
- @include 'my/html/path.html'
- @css 'my/css/path.css'
- @js 'my/js/path.js'
```
## In The Future
As you've understood, the DTM is still under development and optimization. There are
functionalities to be added, such as data compression, managing loops or conditions within the
template itself. Able to be used in contexts other than HTML and raw text.
This will come in time.
Feel free to report any issues or contribute to the project!
File diff suppressed because it is too large. Load diff
@@ -0,0 +1,333 @@
// vtest retry: 3
module dtm
import os
import time
const behavior_temp_dtm_dir = 'dynamic_template_manager_behavior_test'
const behavior_temp_cache_dir = 'vcache_dtm'
const behavior_temp_templates_dir = 'templates'
const behavior_vtmp_dir = os.vtmp_dir()
fn behavior_test_root_dir() string {
return os.join_path(behavior_vtmp_dir, '${behavior_temp_dtm_dir}_${os.getpid()}')
}
fn testsuite_begin() {
temp_folder := behavior_test_root_dir()
os.rmdir_all(temp_folder) or {}
os.mkdir_all(os.join_path(temp_folder, behavior_temp_cache_dir))!
os.mkdir_all(os.join_path(temp_folder, behavior_temp_templates_dir))!
templates_path := os.join_path(temp_folder, behavior_temp_templates_dir)
os.write_file(os.join_path(templates_path, 'page.html'), [
'<!doctype html>',
'<title>@title</title>',
'<p>@body</p>',
'<p>@count</p>',
].join('\n'))!
os.write_file(os.join_path(templates_path, 'page.txt'), [
'Title: @title',
'Body: @body',
].join('\n'))!
os.write_file(os.join_path(templates_path, 'layout.html'), [
'<header>Start</header>',
"@include 'partial'",
'<footer>End</footer>',
].join('\n'))!
os.write_file(os.join_path(templates_path, 'partial.html'), '<section>@title</section>')!
os.write_file(os.join_path(templates_path, 'cache.html'), '<main>@title</main>')!
os.write_file(os.join_path(templates_path, 'cache_file_update.html'), '<main>@title</main>')!
os.write_file(os.join_path(templates_path, 'no_store.html'), '<main>@title</main>')!
os.write_file(os.join_path(templates_path, 'forever.html'), '<main>@title</main>')!
os.write_file(os.join_path(templates_path, 'disk_cache.html'), '<main>@title</main>')!
os.write_file(os.join_path(templates_path, 'include_html.html'), '<article>@content</article>')!
os.write_file(os.join_path(templates_path, 'include_text.txt'), 'Text: @content')!
os.write_file(os.join_path(templates_path, 'invalid.css'), 'body { color: red; }')!
}
fn testsuite_end() {
os.rmdir_all(behavior_test_root_dir()) or {}
}
fn new_behavior_dtm(active_cache_server bool, compress_html bool, max_size_data_in_mem int) &DynamicTemplateManager {
temp_folder := behavior_test_root_dir()
return initialize(
active_cache_server: active_cache_server
compress_html: compress_html
max_size_data_in_mem: max_size_data_in_mem
test_cache_dir: os.join_path(temp_folder, behavior_temp_cache_dir)
test_template_dir: os.join_path(temp_folder, behavior_temp_templates_dir)
)
}
fn behavior_template_path(file_name string) string {
return os.join_path(behavior_test_root_dir(), behavior_temp_templates_dir, file_name)
}
fn rendered_cache_count(dtmi &DynamicTemplateManager) int {
rlock dtmi.template_caches {
return dtmi.template_caches.len
}
}
fn test_expand_escapes_html_placeholders_by_default() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'title': DtmMultiTypeMap('Hello <V>')
'body': DtmMultiTypeMap('<script>alert(1)</script>')
'count': DtmMultiTypeMap(7)
}
rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
assert rendered.contains('<title>Hello &lt;V&gt;</title>')
assert rendered.contains('<p>&lt;script&gt;alert(1)&lt;/script&gt;</p>')
assert rendered.contains('<p>7</p>')
}
fn test_expand_does_not_double_escape_placeholder_values_containing_dollar() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'title': DtmMultiTypeMap('Cost $5 <V>')
'body': DtmMultiTypeMap('Body')
'count': DtmMultiTypeMap(1)
}
rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
assert rendered.contains('<title>Cost $5 &lt;V&gt;</title>')
}
fn test_expand_escapes_html_entities_and_quotes() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'title': DtmMultiTypeMap('A & B "quoted" \'single\'')
'body': DtmMultiTypeMap('Body')
'count': DtmMultiTypeMap(1)
}
rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
assert rendered.contains('<title>A &amp; B &#34;quoted&#34; &#39;single&#39;</title>')
}
fn test_expand_escapes_text_templates() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'title': DtmMultiTypeMap('Plain <Title>')
'body': DtmMultiTypeMap('<strong>raw text</strong>')
}
rendered := dtmi.expand(behavior_template_path('page.txt'), placeholders: &placeholders)
assert rendered.contains('Title: Plain &lt;Title&gt;')
assert rendered.contains('Body: &lt;strong&gt;raw text&lt;/strong&gt;')
}
fn test_expand_compresses_html_without_regex_failure() {
mut dtmi := new_behavior_dtm(false, true, max_size_data_in_memory)
placeholders := {
'title': DtmMultiTypeMap('Compressed')
'body': DtmMultiTypeMap('Body')
'count': DtmMultiTypeMap(1)
}
rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
assert rendered != internat_server_error
assert !rendered.contains('\n')
assert rendered.contains('<!doctype html><title>Compressed</title><p>Body</p><p>1</p>')
}
fn test_expand_with_empty_placeholders_keeps_template_renderable() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := map[string]DtmMultiTypeMap{}
rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
assert rendered.contains('<title>$title</title>')
assert rendered.contains('<p>$body</p>')
assert rendered.contains('<p>$count</p>')
}
fn test_expand_resolves_relative_includes() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'title': DtmMultiTypeMap('Included <Title>')
}
rendered := dtmi.expand(behavior_template_path('layout.html'), placeholders: &placeholders)
assert rendered.contains('<header>Start</header>')
assert rendered.contains('<section>Included &lt;Title&gt;</section>')
assert rendered.contains('<footer>End</footer>')
}
fn test_includehtml_preserves_allowed_tags_and_escapes_unlisted_tags() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'content_#includehtml': DtmMultiTypeMap('<span>allowed</span><script>blocked()</script>')
}
rendered :=
dtmi.expand(behavior_template_path('include_html.html'), placeholders: &placeholders)
assert rendered.contains('<article><span>allowed</span>&lt;script&gt;blocked()&lt;/script&gt;</article>')
}
fn test_includehtml_is_still_escaped_in_text_templates() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := {
'content_#includehtml': DtmMultiTypeMap('<span>text</span>')
}
rendered := dtmi.expand(behavior_template_path('include_text.txt'), placeholders: &placeholders)
assert rendered.contains('Text: &lt;span&gt;text&lt;/span&gt;')
}
fn test_expand_returns_internal_error_for_missing_template() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := map[string]DtmMultiTypeMap{}
rendered := dtmi.expand(behavior_template_path('missing.html'), placeholders: &placeholders)
assert rendered == internat_server_error
}
fn test_expand_returns_internal_error_for_invalid_template_extension() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
placeholders := map[string]DtmMultiTypeMap{}
rendered := dtmi.expand(behavior_template_path('invalid.css'), placeholders: &placeholders)
assert rendered == internat_server_error
}
fn test_expand_without_cache_renders_fresh_placeholder_content() {
mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
first_placeholders := {
'title': DtmMultiTypeMap('first')
}
second_placeholders := {
'title': DtmMultiTypeMap('second')
}
first := dtmi.expand(behavior_template_path('cache.html'), placeholders: &first_placeholders)
second := dtmi.expand(behavior_template_path('cache.html'), placeholders: &second_placeholders)
assert first.contains('<main>first</main>')
assert second.contains('<main>second</main>')
}
fn test_expand_with_cache_updates_when_placeholder_content_changes() {
mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
defer {
dtmi.disable_cache()
}
first_placeholders := {
'title': DtmMultiTypeMap('cached first')
}
second_placeholders := {
'title': DtmMultiTypeMap('cached second')
}
first := dtmi.expand(behavior_template_path('cache.html'), placeholders: &first_placeholders)
second := dtmi.expand(behavior_template_path('cache.html'), placeholders: &first_placeholders)
third := dtmi.expand(behavior_template_path('cache.html'), placeholders: &second_placeholders)
assert first.contains('<main>cached first</main>')
assert second == first
assert third.contains('<main>cached second</main>')
}
fn test_expand_with_legacy_cache_enabled_uses_new_parsed_template_engine() {
mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
defer {
dtmi.disable_cache()
}
placeholders := {
'title': DtmMultiTypeMap('stable cached value')
}
first := dtmi.expand(behavior_template_path('cache.html'), placeholders: &placeholders)
second := dtmi.expand(behavior_template_path('cache.html'), placeholders: &placeholders)
assert first == second
assert rendered_cache_count(dtmi) == 0
assert dtmi.render_engine.compiled_template_count() == 1
}
fn test_expand_with_cache_updates_when_template_file_changes() {
mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
defer {
dtmi.disable_cache()
}
template_path := behavior_template_path('cache_file_update.html')
placeholders := {
'title': DtmMultiTypeMap('file driven')
}
first := dtmi.expand(template_path, placeholders: &placeholders)
time.sleep(1100 * time.millisecond)
os.write_file(template_path, '<section>@title</section>')!
second := dtmi.expand(template_path, placeholders: &placeholders)
assert first.contains('<main>file driven</main>')
assert second.contains('<section>file driven</section>')
}
fn test_cache_delay_minus_one_skips_rendered_cache_storage() {
mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
defer {
dtmi.disable_cache()
}
placeholders := {
'title': DtmMultiTypeMap('no store')
}
rendered := dtmi.expand(behavior_template_path('no_store.html'),
placeholders: &placeholders
cache_delay_expiration: -1
)
assert rendered.contains('<main>no store</main>')
assert rendered_cache_count(dtmi) == 0
}
fn test_cache_delay_zero_is_accepted_without_rendered_cache_storage() {
mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
defer {
dtmi.disable_cache()
}
placeholders := {
'title': DtmMultiTypeMap('forever')
}
rendered := dtmi.expand(behavior_template_path('forever.html'),
placeholders: &placeholders
cache_delay_expiration: 0
)
assert rendered.contains('<main>forever</main>')
assert rendered_cache_count(dtmi) == 0
}
fn test_memory_limit_zero_does_not_create_rendered_disk_cache() {
mut dtmi := new_behavior_dtm(true, false, 0)
defer {
dtmi.disable_cache()
}
placeholders := {
'title': DtmMultiTypeMap('disk mode')
}
rendered := dtmi.expand(behavior_template_path('disk_cache.html'), placeholders: &placeholders)
assert rendered.contains('<main>disk mode</main>')
assert rendered_cache_count(dtmi) == 0
for file_name in os.ls(dtmi.template_cache_folder)! {
assert !file_name.ends_with('.cache')
}
}
@@ -0,0 +1,176 @@
// vtest retry: 3
module dtm
import os
import time
const temp_dtm_dir = 'dynamic_template_manager_cache_system_test'
const temp_cache_dir = 'vcache_dtm'
const temp_templates_dir = 'templates'
const temp_html_fp = 'temp.html'
const temp_html_n = 'temp'
const vtmp_dir = os.vtmp_dir()
fn testsuite_begin() {
temp_folder := os.join_path(vtmp_dir, temp_dtm_dir)
os.mkdir_all(temp_folder)!
vcache_path := os.join_path(temp_folder, temp_cache_dir)
templates_path := os.join_path(temp_folder, temp_templates_dir)
os.mkdir_all(vcache_path)!
os.mkdir_all(templates_path)!
temp_html_file := os.join_path(templates_path, temp_html_fp)
html_content := '
<!DOCTYPE html>
<html>
<head>
<title>TEST</title>
</head>
<body>
<div>
<H1>TEST</H1>
</div>
</body>
</html>'
os.write_file(temp_html_file, html_content)!
}
fn test_check_and_clear_cache_files() {
dtmi := init_dtm(false, 0)
check_and_clear_cache_files(dtmi.template_cache_folder)!
count_cache_files := os.ls(dtmi.template_cache_folder)!
assert count_cache_files.len == 0
}
fn test_create_template_cache_and_display() {
mut dtmi := init_dtm(true, max_size_data_in_memory)
defer {
dtmi.stop_cache_handler()
}
html := dtmi.create_cache()
assert html.len > 10
}
fn test_get_cache() {
mut dtmi := init_dtm(true, max_size_data_in_memory)
dtmi.create_cache()
defer {
dtmi.stop_cache_handler()
}
dtm_placeholders := map[string]DtmMultiTypeMap{}
temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
html_mem := dtmi.get_cache(temp_html_n, temp_html_file, &dtm_placeholders)
assert html_mem.len > 10
}
fn test_chandler_clear_specific_cache() {
mut dtmi := init_dtm(true, 0)
defer {
dtmi.stop_cache_handler()
}
dtmi.create_cache()
lock dtmi.template_caches {
cache_file := os.join_path(dtmi.template_cache_folder,
'${dtmi.template_caches[0].name}_${dtmi.template_caches[0].checksum}.cache')
index, is_success := dtmi.chandler_clear_specific_cache(dtmi.template_caches[0].id)
assert is_success == true
assert index == 0
cache_exist := os.exists(cache_file)
assert cache_exist == false
}
}
fn test_cache_timestamp_is_initialized() {
mut dtmi := init_dtm(true, max_size_data_in_memory)
defer {
dtmi.stop_cache_handler()
}
date_to_str := dtmi.c_time.str()
assert date_to_str.len > 10
}
fn test_process_rendered_cache_delete_request() {
mut dtmi := init_dtm(true, max_size_data_in_memory)
defer {
dtmi.stop_cache_handler()
}
dtmi.create_cache()
path_f := os.join_path(dtmi.template_folder, temp_html_fp)
lock dtmi.template_caches {
assert dtmi.template_caches[0].id == 1
assert dtmi.template_caches[0].name == temp_html_n
assert dtmi.template_caches[0].path == path_f
}
dtmi.id_to_handlered = 1
dtmi.ch_cache_handler <- TemplateCache{
id: 1
cache_request: .delete
}
dtmi.sync_cache()
rlock dtmi.template_caches {
assert dtmi.template_caches.len == 0
}
}
fn testsuite_end() {
temp_folder := os.join_path(vtmp_dir, temp_dtm_dir)
os.rmdir_all(temp_folder) or {}
}
// Utilities function :
fn init_dtm(b bool, m int) &DynamicTemplateManager {
temp_folder := os.join_path(vtmp_dir, temp_dtm_dir)
vcache_path := os.join_path(temp_folder, temp_cache_dir)
templates_path := os.join_path(temp_folder, temp_templates_dir)
init_params := DynamicTemplateManagerInitialisationParams{
active_cache_server: b
max_size_data_in_mem: m
test_cache_dir: vcache_path
test_template_dir: templates_path
}
dtm := initialize(init_params)
return dtm
}
fn (mut tm DynamicTemplateManager) create_cache() string {
temp_html_file := os.join_path(tm.template_folder, temp_html_fp)
html_last_mod := os.file_last_mod_unix(temp_html_file)
c_time := get_current_unix_micro_timestamp()
cache_delay_exp := i64(500) * i64(1000000)
placeholder := map[string]DtmMultiTypeMap{}
content_checksum := ''
html := tm.create_template_cache_and_display(.new, html_last_mod, c_time, temp_html_file,
temp_html_n, cache_delay_exp, &placeholder, content_checksum, TemplateType.html)
tm.sync_cache()
return html
}
fn (mut tm DynamicTemplateManager) sync_cache() {
mut count := 0
for {
select {
_ := <-tm.is_ready {
break
}
else {
time.sleep(50 * time.millisecond)
// Wait a total of 2 seconds to check if the data cache is ready
if count >= 40 {
tm.abort_test = true
break
}
count++
}
}
}
}
@@ -0,0 +1,65 @@
module dtm
import x.templating.dtm2
// new_dtm2_render_engine is the compatibility boundary between the historical
// DTM API and the DTM2 runtime template engine. DTM2 remains unaware
// of legacy cache concepts; it only parses templates and renders them.
fn new_dtm2_render_engine(template_dir string, compress_html bool) &dtm2.Manager {
return dtm2.initialize(
template_dir: template_dir
compress_html: compress_html
reload_modified_templates: true
)
}
fn (mut tm DynamicTemplateManager) expand_with_dtm2_render_engine(tmpl_path string, tmpl_var TemplateCacheParams) string {
validate_legacy_placeholder_sizes(tmpl_path, tmpl_var.placeholders) or {
eprintln(err.msg())
return internat_server_error
}
if isnil(tm.render_engine) {
tm.render_engine = new_dtm2_render_engine(tm.template_folder, tm.compress_html)
}
placeholders := convert_legacy_placeholders(tmpl_var.placeholders)
return tm.render_engine.expand(tmpl_path,
placeholders: &placeholders
missing_placeholder_prefix: '$'
)
}
fn validate_legacy_placeholder_sizes(tmpl_path string, placeholders &map[string]DtmMultiTypeMap) ! {
for key, value in placeholders {
if key.len > max_placeholders_key_size {
return error('${message_signature_error} Length of placeholder key "${key}" exceeds the maximum allowed size for template content in file: ${tmpl_path}. Max allowed size: ${max_placeholders_key_size} characters.')
}
casted_value := legacy_placeholder_value_to_string(*value)
if casted_value.len > max_placeholders_value_size {
return error('${message_signature_error} Length of placeholder value for key "${key}" exceeds the maximum allowed size for template content in file: ${tmpl_path}. Max allowed size: ${max_placeholders_value_size} characters.')
}
}
}
fn convert_legacy_placeholders(placeholders &map[string]DtmMultiTypeMap) map[string]string {
mut converted := map[string]string{}
for key, value in placeholders {
converted[key.clone()] = legacy_placeholder_value_to_string(*value).clone()
}
return converted
}
fn legacy_placeholder_value_to_string(value DtmMultiTypeMap) string {
return match value {
f32 { value.str() }
f64 { value.str() }
i16 { value.str() }
i64 { value.str() }
i8 { value.str() }
int { value.str() }
string { value.clone() }
u16 { value.str() }
u32 { value.str() }
u64 { value.str() }
u8 { value.str() }
}
}
@@ -0,0 +1,342 @@
module dtm
import os
const temp_dtm_dir = 'dynamic_template_manager_test'
const temp_templates_dir = 'templates'
const temp_html_fp = 'temp.html'
const temp_html_n = 'temp'
const vtmp_dir = os.vtmp_dir()
fn dtm_test_root_dir() string {
return os.join_path(vtmp_dir, '${temp_dtm_dir}_${os.getpid()}')
}
fn testsuite_begin() {
temp_folder := dtm_test_root_dir()
os.rmdir_all(temp_folder) or {}
os.mkdir_all(temp_folder)!
templates_path := os.join_path(temp_folder, temp_templates_dir)
os.mkdir_all(templates_path)!
temp_html_file := os.join_path(templates_path, temp_html_fp)
html_content := '
<!DOCTYPE html>
<html>
<head>
<title>TEST</title>
</head>
<body>
<div>
<H1>TEST</H1>
</div>
</body>
</html>'
os.write_file(temp_html_file, html_content)!
}
fn test_initialize_dtm() {
dtmi := init_dtm(false, 0)
assert dtmi.dtm_init_is_ok == true
}
fn test_create_template_cache_and_display() {
mut dtmi := init_dtm(false, 0)
temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
html_last_mod := os.file_last_mod_unix(temp_html_file)
c_time := get_current_unix_micro_timestamp()
cache_delay_exp := i64(500) * i64(1000000)
placeholder := map[string]DtmMultiTypeMap{}
content_checksum := ''
html := dtmi.create_template_cache_and_display(.new, html_last_mod, c_time, temp_html_file,
temp_html_n, cache_delay_exp, &placeholder, content_checksum, TemplateType.html)
assert html.len > 10
}
fn test_return_cache_info_isexistent() {
mut dtmi := init_dtm(false, 0)
path_template := os.join_path(dtmi.template_folder, temp_html_fp)
lock dtmi.template_caches {
dtmi.template_caches << TemplateCache{
id: 1
path: path_template
}
}
lock dtmi.nbr_of_remaining_template_request {
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 1
}
}
cache_exists, _, _, _, _, _, _ := dtmi.return_cache_info_isexistent(path_template)
assert cache_exists == true
lock dtmi.template_caches {
dtmi.template_caches[0].id_redirection = 2
dtmi.template_caches << TemplateCache{
id: 2
path: path_template
id_redirection: 3
}
dtmi.template_caches << TemplateCache{
id: 3
path: path_template
id_redirection: 4
}
dtmi.template_caches << TemplateCache{
id: 4
path: path_template
id_redirection: 5
}
dtmi.template_caches << TemplateCache{
id: 5
path: path_template
}
}
lock dtmi.nbr_of_remaining_template_request {
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 2
}
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 3
}
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 4
}
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 5
}
}
_, id, _, _, _, _, _ := dtmi.return_cache_info_isexistent(path_template)
assert id == 5
}
fn test_remaining_template_request() {
mut dtmi := init_dtm(false, 0)
lock dtmi.nbr_of_remaining_template_request {
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 1
}
}
dtmi.remaining_template_request(true, 1)
rlock dtmi.nbr_of_remaining_template_request {
assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 1
}
dtmi.remaining_template_request(true, 1)
rlock dtmi.nbr_of_remaining_template_request {
assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 2
}
dtmi.remaining_template_request(false, 1)
rlock dtmi.nbr_of_remaining_template_request {
assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 1
}
dtmi.remaining_template_request(false, 1)
rlock dtmi.nbr_of_remaining_template_request {
assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 0
}
}
fn test_check_tmpl_and_placeholders_size() {
mut dtmi := init_dtm(false, 0)
temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
placeholders := map[string]DtmMultiTypeMap{}
path, filename, content_checksum, tmpl_type := dtmi.check_tmpl_and_placeholders_size(temp_html_file,
&placeholders)!
is_html := if tmpl_type == TemplateType.html {
true
} else {
false
}
assert path.len > 10
assert filename.len > 3
assert is_html == true
// assert content_checksum.len > 3
}
fn test_chandler_prevent_cache_duplicate_request() {
mut dtmi := init_dtm(false, 0)
temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
lock dtmi.template_caches {
dtmi.template_caches << TemplateCache{
id: 1
path: temp_html_file
cache_request: .new
}
dtmi.template_caches << TemplateCache{
id: 2
path: temp_html_file
cache_request: .update
last_template_mod: i64(1)
}
dtmi.template_caches << TemplateCache{
id: 3
path: temp_html_file
cache_request: .exp_update
last_template_mod: i64(1)
generate_at: i64(100)
}
dtmi.template_caches << TemplateCache{
id: 4
cache_request: .delete
}
}
new_cache := TemplateCache{
id: 5
path: temp_html_file
cache_request: .new
}
update_cache := TemplateCache{
id: 6
path: temp_html_file
cache_request: .update
last_template_mod: i64(1)
}
exp_update_cache := TemplateCache{
id: 7
path: temp_html_file
cache_request: .exp_update
last_template_mod: i64(1)
generate_at: i64(10)
cache_delay_expiration: i64(10)
}
delete_cache := TemplateCache{
id: 4
cache_request: .delete
}
mut is_duplicate := dtmi.chandler_prevent_cache_duplicate_request(&new_cache)
assert is_duplicate == true
is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&update_cache)
assert is_duplicate == true
is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&exp_update_cache)
assert is_duplicate == true
is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&delete_cache)
assert is_duplicate == false
lock dtmi.template_caches {
dtmi.template_caches.delete(3)
}
is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&delete_cache)
assert is_duplicate == true
}
fn test_chandler_remaining_cache_template_used() {
mut dtmi := init_dtm(false, 0)
lock dtmi.nbr_of_remaining_template_request {
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 1
nbr_of_remaining_request: 0
}
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 2
nbr_of_remaining_request: 1
need_to_delete: true
}
dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
id: 3
nbr_of_remaining_request: 0
need_to_delete: true
}
}
mut can_delete := dtmi.chandler_remaining_cache_template_used(CacheRequest.update, 3, 3)
assert can_delete == true
can_delete = dtmi.chandler_remaining_cache_template_used(CacheRequest.update, 2, 2)
assert can_delete == false
can_delete = dtmi.chandler_remaining_cache_template_used(CacheRequest.delete, 1, 0)
assert can_delete == true
can_delete = dtmi.chandler_remaining_cache_template_used(CacheRequest.new, 4, 0)
assert can_delete == true
}
fn test_parse_tmpl_file() {
mut dtmi := init_dtm(false, 0)
temp_folder := dtm_test_root_dir()
templates_path := os.join_path(temp_folder, temp_templates_dir)
temp_html_file := os.join_path(templates_path, temp_html_fp)
mut placeholders := map[string]DtmMultiTypeMap{}
is_compressed := true
html := dtmi.parse_tmpl_file(temp_html_file, temp_html_n, &placeholders, is_compressed,
TemplateType.html)
assert html.len > 0
}
fn test_check_if_cache_delay_iscorrect() {
check_if_cache_delay_iscorrect(i64(300 * 1000000), temp_html_n) or { assert false }
check_if_cache_delay_iscorrect(i64(-100), temp_html_n) or { return }
assert false
}
fn test_cache_request_route() {
mut dtmi := init_dtm(false, 0)
mut is_cache_exist := true
mut cache_delay_expiration := i64(400)
mut last_template_mod := get_current_unix_micro_timestamp()
mut test_current_template_mod := last_template_mod
mut cache_del_exp := 300
mut gen_at := last_template_mod
mut content_checksum := 'checksumtest1'
mut current_content_checksum := 'checksumtest2'
mut request_type, _ := dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
assert request_type == CacheRequest.update
current_content_checksum = 'checksumtest1'
request_type, _ = dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
assert request_type == CacheRequest.cached
gen_at = (last_template_mod - 500)
request_type, _ = dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
assert request_type == CacheRequest.exp_update
is_cache_exist = false
request_type, _ = dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
assert request_type == CacheRequest.new
}
fn testsuite_end() {
temp_folder := dtm_test_root_dir()
os.rmdir_all(temp_folder) or {}
}
// Utilities function :
fn init_dtm(b bool, m int) &DynamicTemplateManager {
temp_folder := dtm_test_root_dir()
templates_path := os.join_path(temp_folder, temp_templates_dir)
init_params := DynamicTemplateManagerInitialisationParams{
active_cache_server: b
max_size_data_in_mem: m
test_template_dir: templates_path
}
dtm := initialize(init_params)
return dtm
}
@@ -0,0 +1,30 @@
module dtm
import strings
fn filter(s string) string {
mut escaped := strings.new_builder(s.len)
for i := 0; i < s.len; i++ {
match s[i] {
`&` {
escaped.write_string('&amp;')
}
`<` {
escaped.write_string('&lt;')
}
`>` {
escaped.write_string('&gt;')
}
`"` {
escaped.write_string('&#34;')
}
`'` {
escaped.write_string('&#39;')
}
else {
escaped.write_u8(s[i])
}
}
}
return escaped.str()
}
+358
View File
@@ -0,0 +1,358 @@
// Copyright (c) 2019-2024 Alexander Medvednikov. All rights reserved.
// Use of this source code is governed by an MIT license
// that can be found in the LICENSE file.
/*
This source code originates from the internal V compiler 'vlib/v/parser/tmpl.v' and
has been heavily modified for the needs of the Dynamic Template Manager. Thanks to its original author, Alexander Medvednikov.
*/
module dtm
import os
import strings
enum State {
simple // default - no special interpretation of tags, *at all*!
// That is suitable for the general case of text template interpolation,
// for example for interpolating arbitrary source code (even V source) templates.
html // default, only when the template extension is .html
css // <style>
js // <script>
// span // span.{
}
fn (mut state State) update(line string) {
trimmed_line := line.trim_space()
if is_html_open_tag('style', line) {
state = .css
} else if trimmed_line == '</style>' {
state = .html
} else if is_html_open_tag('script', line) {
state = .js
} else if trimmed_line == '</script>' {
state = .html
}
}
const tmpl_str_end = "')\n"
// check HTML open tag `<name attr="x" >`
fn is_html_open_tag(name string, s string) bool {
trimmed_line := s.trim_space()
mut len := trimmed_line.len
if len < name.len {
return false
}
mut sub := trimmed_line[0..1]
if sub != '<' { // not start with '<'
return false
}
sub = trimmed_line[len - 1..len]
if sub != '>' { // not end with '<'
return false
}
sub = trimmed_line[len - 2..len - 1]
if sub == '/' { // self-closing
return false
}
sub = trimmed_line[1..len - 1]
if sub.contains_any('<>') { // `<name <bad> >`
return false
}
if sub == name { // `<name>`
return true
} else {
len = name.len
if sub.len <= len { // `<nam>` or `<meme>`
return false
}
if sub[..len + 1] != '${name} ' { // not `<name ...>`
return false
}
return true
}
}
fn replace_placeholders_with_data(line string, data &map[string]DtmMultiTypeMap, state State) string {
mut rline := line
if data.len == 0 {
return rline
}
mut need_include_html := false
for key, value in data {
mut placeholder := '$${key}'
if placeholder.ends_with(include_html_key_tag) {
placeholder = placeholder.all_before_last(include_html_key_tag)
need_include_html = true
}
if !rline.contains(placeholder) {
need_include_html = false
continue
}
mut val_str := ''
match value {
i8, i16, int, i64, u8, u16, u32, u64, f32, f64 {
// Converts value to string
temp_val := value.str()
// Filters the string value for safe insertion
val_str = filter(temp_val)
}
string {
// Checks if the placeholder allows HTML inclusion
if need_include_html {
if state == State.html {
// Escape the whole value first, then restore only the explicit allow-list.
// This preserves the documented opt-in HTML behavior without allowing
// arbitrary raw tags through `_#includehtml`.
val_str = filter(value)
for tag in allowed_tags {
escaped_tag := filter(tag)
val_str = val_str.replace(escaped_tag, tag)
}
} else {
val_str = filter(value)
}
need_include_html = false
} else {
// Filters the string value for safe insertion
val_str = filter(value)
}
}
}
rline = rline.replace(placeholder, val_str)
}
return rline
}
fn insert_template_code(fn_name string, tmpl_str_start string, line string, data &map[string]DtmMultiTypeMap,
state State) string {
// HTML, may include `@var`
// escaped by cgen, unless it's a `veb.RawHtml` string
trailing_bs := tmpl_str_end + 'sb_${fn_name}.write_u8(92)\n' + tmpl_str_start
mut rline := line.replace('\\', '\\\\').replace("'", "\\'").replace('@', '$')
rline = rline.replace(r'$$', r'\@').replace(r'.$', r'.@')
comptime_call_str := rline.find_between('\${', '}')
if comptime_call_str.contains("\\'") {
rline = rline.replace(comptime_call_str, comptime_call_str.replace("\\'", r"'"))
}
if rline.ends_with('\\') {
rline = rline[0..rline.len - 2] + trailing_bs
}
if rline.contains('$') {
rline = replace_placeholders_with_data(rline, data, state)
}
return rline
}
// compile_file compiles the content of a file by the given path as a template
fn compile_template_file(template_file string, fn_name string, data &map[string]DtmMultiTypeMap) string {
template_content := os.read_file(template_file) or {
eprintln('${message_signature_error} Template generator can not reading from ${template_file} file')
return internat_server_error
}
mut lines := template_content.split_into_lines()
basepath := os.dir(template_file)
tmpl_str_start := "\tsb_${fn_name}.write_string('"
mut source := strings.new_builder(1000)
mut state := State.simple
template_ext := os.file_ext(template_file)
if template_ext.to_lower() == '.html' {
state = .html
}
mut in_span := false
mut end_of_line_pos := 0
mut start_of_line_pos := 0
mut tline_number := -1 // keep the original line numbers, even after insert/delete ops on lines; `i` changes
for i := 0; i < lines.len; i++ {
line := lines[i]
tline_number++
start_of_line_pos = end_of_line_pos
end_of_line_pos += line.len + 1
if state != .simple {
state.update(line)
}
$if trace_tmpl ? {
eprintln('>>> tfile: ${template_file}, spos: ${start_of_line_pos:6}, epos:${end_of_line_pos:6}, fi: ${tline_number:5}, i: ${i:5}, state: ${state:10}, line: ${line}')
}
if line.contains('@header') {
position := line.index('@header') or { 0 }
eprintln("${message_signature_warn} Please use @include 'header' instead of @header (deprecated), position : ${position}")
continue
}
if line.contains('@footer') {
position := line.index('@footer') or { 0 }
eprintln("${message_signature_warn} Please use @include 'footer' instead of @footer (deprecated), position : ${position}")
continue
}
if line.contains('@include ') {
lines.delete(i)
// Allow single or double quoted paths.
mut file_name := if line.contains('"') {
line.split('"')[1]
} else if line.contains("'") {
line.split("'")[1]
} else {
s := '@include '
position := line.index(s) or { 0 }
eprintln("${message_signature_error} path for @include must be quoted with ' or \" without line breaks or extraneous characters between @include and the quotes, position : ${position}")
return internat_server_error
}
mut file_ext := os.file_ext(file_name)
if file_ext == '' {
file_ext = '.html'
}
file_name = file_name.replace(file_ext, '')
// relative path, starting with the current folder
mut templates_folder := os.real_path(basepath)
if file_name.contains('/') && file_name.starts_with('/') {
// an absolute path
templates_folder = ''
}
file_path := os.real_path(os.join_path_single(templates_folder,
'${file_name}${file_ext}'))
$if trace_tmpl ? {
eprintln('>>> basepath: "${basepath}" , template_file: "${template_file}" , fn_name: "${fn_name}" , @include line: "${line}" , file_name: "${file_name}" , file_ext: "${file_ext}" , templates_folder: "${templates_folder}" , file_path: "${file_path}"')
}
file_content := os.read_file(file_path) or {
position := line.index('@include ') or { 0 } + '@include '.len
eprintln('${message_signature_error} Reading @include file "${file_name}" from path: ${file_path} failed, position : ${position}')
return internat_server_error
}
file_splitted := file_content.split_into_lines().reverse()
for f in file_splitted {
tline_number--
lines.insert(i, f)
}
i--
continue
}
if line.contains('@if ') {
/* source.writeln(dtm.tmpl_str_end)
pos := line.index('@if') or { continue }
source.writeln('if ' + line[pos + 4..] + '{')
source.writeln(tmpl_str_start) */
continue
}
if line.contains('@end') {
/* // Remove new line byte
source.go_back(1)
source.writeln(dtm.tmpl_str_end)
source.writeln('}')
source.writeln(tmpl_str_start) */
continue
}
if line.contains('@else') {
// Remove new line byte
source.go_back(1)
/* source.writeln(dtm.tmpl_str_end)
source.writeln(' } else { ')
source.writeln(tmpl_str_start) */
continue
}
if line.contains('@for') {
/* source.writeln(dtm.tmpl_str_end)
pos := line.index('@for') or { continue }
source.writeln('for ' + line[pos + 4..] + '{')
source.writeln(tmpl_str_start) */
continue
}
if state == .simple {
// by default, just copy 1:1
source.writeln(insert_template_code(fn_name, tmpl_str_start, line, data, state))
continue
}
// The .simple mode ends here. The rest handles .html/.css/.js state transitions.
if state != .simple {
if line.contains('@js ') {
pos := line.index('@js') or { continue }
source.write_string('<script src="')
source.write_string(line[pos + 5..line.len - 1])
source.writeln('"></script>')
continue
}
if line.contains('@css ') {
pos := line.index('@css') or { continue }
source.write_string('<link href="')
source.write_string(line[pos + 6..line.len - 1])
source.writeln('" rel="stylesheet" type="text/css">')
continue
}
}
match state {
.html {
line_t := line.trim_space()
if line_t.starts_with('span.') && line.ends_with('{') {
//`span.header {` => `<span class='header'>`
class := line.find_between('span.', '{').trim_space()
source.writeln('<span class="${class}">')
in_span = true
continue
} else if line_t.starts_with('.') && line.ends_with('{') {
//`.header {` => `<div class='header'>`
class := line.find_between('.', '{').trim_space()
trimmed := line.trim_space()
source.write_string(strings.repeat(`\t`, line.len - trimmed.len)) // add the necessary indent to keep <div><div><div> code clean
source.writeln('<div class="${class}">')
continue
} else if line_t.starts_with('#') && line.ends_with('{') {
//`#header {` => `<div id='header'>`
class := line.find_between('#', '{').trim_space()
source.writeln('<div id="${class}">')
continue
} else if line_t == '}' {
source.write_string(strings.repeat(`\t`, line.len - line_t.len)) // add the necessary indent to keep <div><div><div> code clean
if in_span {
source.writeln('</span>')
in_span = false
} else {
source.writeln('</div>')
}
continue
}
}
.js {
// if line.contains('//V_TEMPLATE') {
source.writeln(insert_template_code(fn_name, tmpl_str_start, line, data, state))
//} else {
// replace `$` to `\$` at first to escape JavaScript template literal syntax
// source.writeln(line.replace(r'$', r'\$').replace(r'$$', r'@').replace(r'.$',
// r'.@').replace(r"'", r"\'"))
//}
continue
}
.css {
// disable template variable declaration in inline stylesheet
// because of some CSS rules prefixed with `@`.
source.writeln(line.replace(r'.$', r'.@').replace(r"'", r"\'"))
continue
}
else {}
}
// by default, just copy 1:1
source.writeln(insert_template_code(fn_name, tmpl_str_start, line, data, state))
}
result := source.str()
$if trace_tmpl_expansion ? {
eprintln('>>>>>>> template expanded to:')
eprintln(result)
eprintln('-----------------------------')
}
return result
}
+366
View File
@@ -0,0 +1,366 @@
# dtm2 - Dynamic Template Manager 2
`dtm2` is the modern runtime renderer for Dynamic Template Manager.
It keeps the original DTM idea: templates are normal files on disk, so they can
be edited without recompiling the application.
The main change from `x.templating.dtm` is architectural. `dtm2` caches parsed
template trees, not rendered HTML responses. This keeps rendering fast while
removing the old async rendered-cache server from the hot path.
## Quick Start
Create a `templates/` folder in your application and put your templates inside
it. DTM2 supports `.html`, `.htm`, `.xml`, `.txt`, and `.text` by default.
```v
import x.templating.dtm2
fn main() {
mut manager := dtm2.initialize(
template_dir: 'templates'
)
placeholders := {
'title': 'DTM2'
'body': '<strong>escaped by default</strong>'
}
rendered := manager.expand('page.html', placeholders: &placeholders)
println(rendered)
}
```
Example template:
```html
<!doctype html>
<html>
<head>
<title>@title</title>
</head>
<body>
<main>@body</main>
</body>
</html>
```
The rendered `@body` value is escaped by default.
## Veb Example
```v
import veb
import x.templating.dtm2
pub struct App {
pub mut:
templates &dtm2.Manager = unsafe { nil }
}
pub struct Context {
veb.Context
}
fn main() {
mut app := &App{
templates: dtm2.initialize(
template_dir: 'templates'
)
}
veb.run[App, Context](mut app, 18081)
}
@['/']
pub fn (mut app App) index(mut ctx Context) veb.Result {
placeholders := {
'title': 'Home'
'body': 'Hello from DTM2'
}
html := app.templates.expand('index.html', placeholders: &placeholders)
return ctx.html(html)
}
```
## Available Options
### Manager options
`dtm2.initialize()` accepts:
- `template_dir` (**string**): root directory used for relative template paths.
If empty, `<executable directory>/templates` is used.
- `compress_html` (**bool**): enables a lightweight deterministic HTML
whitespace compressor. It is enabled by default.
- `reload_modified_templates` (**bool**): when enabled, DTM2 checks the source
template and included files before reusing a parsed template tree. It is
enabled by default.
- `extension_config_file` (**string**): optional JSON file containing extension
mappings. If empty, DTM2 automatically loads `dtm2_extensions.json` from the
configured `template_dir` when that file exists.
Example:
```v ignore
mut manager := dtm2.initialize(
template_dir: 'templates'
compress_html: true
reload_modified_templates: true
)
```
For maximum hot-path throughput in applications where templates are immutable
after startup, you can disable reload checks:
```v ignore
mut manager := dtm2.initialize(
template_dir: 'templates'
reload_modified_templates: false
)
```
The recommended runtime model is one long-lived manager per application or
rendering context. Reusing the manager is what keeps parsed templates and path
resolution cached.
### Template extensions
DTM2 has two rendering modes:
- `TemplateType.html`: HTML/XML-like output, with default escaping and optional
HTML compression.
- `TemplateType.text`: raw text output, also escaped by default.
Default mappings:
- HTML mode: `.html`, `.htm`, `.xml`
- Text mode: `.txt`, `.text`
Project-specific extensions should be configured with a JSON file.
Example `templates/dtm2_extensions.json`:
```json
{
"html": [".view", ".tmpl"],
"text": [".mail", ".md"]
}
```
If the file is named `dtm2_extensions.json` and is placed directly in the
configured `template_dir`, DTM2 loads it automatically:
```v ignore
mut manager := dtm2.initialize(
template_dir: 'templates'
)
```
You can also point to an explicit config file:
```v ignore
mut manager := dtm2.initialize(
template_dir: 'templates'
extension_config_file: 'config/my_dtm2_extensions.json'
)
```
DTM2 ships a default config example that can be copied into your own
`templates/` directory:
```sh
vlib/x/templating/dtm2/dtm2_extensions.json
```
If the JSON file is absent, invalid, or contains invalid entries, DTM2 keeps the
built-in defaults and prints a warning for entries that cannot be registered.
JSON extension config files are limited to 64 KB and every extension is
validated before being registered.
### Render options
`manager.expand()` accepts:
- `placeholders` (**&map[string]string**): values inserted in the template.
- `missing_placeholder_prefix` (**string**): prefix written when a placeholder
is missing. The default is `@`, preserving the original placeholder text.
Example:
```v ignore
placeholders := {
'title': 'Example'
}
html := manager.expand('page.html',
placeholders: &placeholders
missing_placeholder_prefix: '@'
)
```
## The Placeholder System
Template placeholders use the `@name` form.
```html
<h1>@title</h1>
<p>@body</p>
```
In DTM2, placeholder values are strings:
```v ignore
placeholders := {
'title': 'Hello'
'count': '42'
}
```
Values are escaped by default in both HTML and text templates.
<u>**Security note for custom non-HTML formats such as SQL:** DTM2 is a template
renderer, not a domain-specific sanitizer. By default, it escapes HTML-special
characters in placeholder values. It does not make SQL queries safe, does not
replace prepared statements, and does not validate business-specific formats.
If you add `.sql` or another sensitive extension through configuration, the
security of that generated content remains the responsibility of the
application.</u>
## Explicit HTML Inclusion
The historical `_#includehtml` suffix is still supported for compatibility.
It allows a placeholder to include a restricted set of HTML tags in `.html`
templates.
```v ignore
placeholders := {
'body_#includehtml': '<p>allowed</p><script>escaped</script>'
}
html := manager.expand('page.html', placeholders: &placeholders)
```
The template still uses the normal placeholder name:
```html
<main>@body</main>
```
DTM2 escapes the complete value first, then restores only allowed tags. This
preserves the old opt-in behavior without allowing arbitrary raw HTML through.
In `.txt` templates, HTML is always escaped.
Allowed tags:
```html
<div>, </div>, <h1>, </h1>, <h2>, </h2>, <h3>, </h3>, <h4>, </h4>,
<h5>, </h5>, <h6>, </h6>, <p>, </p>, <br>, <hr>, <span>, </span>,
<ul>, </ul>, <ol>, </ol>, <li>, </li>, <dl>, </dl>, <dt>, </dt>,
<dd>, </dd>, <menu>, </menu>, <table>, </table>, <caption>, </caption>,
<th>, </th>, <tr>, </tr>, <td>, </td>, <thead>, </thead>,
<thread>, </thread>, <tbody>, </tbody>, <tfoot>, </tfoot>, <col>, </col>,
<colgroup>, </colgroup>, <header>, </header>, <footer>, </footer>,
<main>, </main>, <section>, </section>, <article>, </article>,
<aside>, </aside>, <details>, </details>, <dialog>, </dialog>,
<data>, </data>, <summary>, </summary>
```
## Includes
Templates can include other templates with a simple line-level directive:
```html
<header>@include "partials/nav"</header>
<main>@body</main>
```
Include paths are resolved relative to the current template. If no file
extension is provided, `.html` is added. The final resolved include path must
stay inside the manager `template_dir`; attempts to include files through `../`
or absolute paths outside that root fail.
The same boundary applies to templates passed to `expand()`. Absolute template
paths are accepted only when they resolve inside `template_dir`.
Included files are tracked as dependencies of the parsed template. When
`reload_modified_templates` is enabled, changing an included file invalidates
the cached parsed tree.
## Backward Compatibility With DTM v1
Existing code that imports `x.templating.dtm` is kept source-compatible for the
migration period. The v1 facade is deprecated, but it now delegates rendering to
DTM2 internally.
That means old code can continue to compile:
```v ignore
import x.templating.dtm
mut manager := dtm.initialize()
mut placeholders := map[string]dtm.DtmMultiTypeMap{}
placeholders['title'] = 'Legacy DTM'
placeholders['count'] = 7
html := manager.expand('page.html', placeholders: &placeholders)
```
For new code, prefer importing `x.templating.dtm2` directly:
```v ignore
import x.templating.dtm2
mut manager := dtm2.initialize(template_dir: 'templates')
placeholders := {
'title': 'Modern DTM'
'count': '7'
}
html := manager.expand('page.html', placeholders: &placeholders)
```
Migration notes:
- Replace `import x.templating.dtm` with `import x.templating.dtm2`.
- Replace `DtmMultiTypeMap` placeholder maps with `map[string]string`.
- Convert numeric values to strings before rendering.
- Remove `stop_cache_handler()` calls; DTM2 does not start an async cache
server.
- Keep using `_#includehtml` only when HTML inclusion is intentional.
## Design Notes
DTM2 intentionally keeps rendering and rendered-output caching separate.
The manager caches:
- canonical template paths;
- parsed template trees;
- dependency metadata for root templates and includes.
The manager does not cache rendered HTML responses. If a future rendered-cache
layer is needed, it should remain a small optional layer above DTM2 rather than
part of the parser/renderer core.
## Benchmarks
The local benchmark harness lives in:
```sh
vlib/x/templating/dtm2/benchmarks/
```
Run it from the repository root:
```sh
vlib/x/templating/dtm2/benchmarks/run_dtm2_benchmark.sh
```
Useful options:
- `DTM2_BENCH_MODE=prod|prod_o2|dev`
- `DTM2_BENCH_CASE=all|small_hot|small_cold|many_hot|many_cold|include_hot`
- `DTM2_BENCH_CASE=include_cold|xml_hot|xml_cold`
- `DTM2_BENCH_ITERATIONS=50000`
- `DTM2_BENCH_COLD_ITERATIONS=500`
- `DTM2_BENCH_PLACEHOLDERS=50`
- `DTM2_BENCH_COMPRESS_HTML=true`
- `DTM2_BENCH_RELOAD_MODIFIED_TEMPLATES=false`
- `DTM2_BENCH_VALIDATE_EACH_ITERATION=false`
Benchmark result directories are generated locally under
`vlib/x/templating/dtm2/benchmarks/results/` and should not be committed.
@@ -0,0 +1,212 @@
module main
import os
import strings
import time
import x.templating.dtm2
const bench_root_name = 'dtm2_benchmark'
struct BenchConfig {
case_name string
iterations int
cold_iterations int
placeholder_count int
compress_html bool
reload_modified_templates bool
validate_each_iteration bool
}
fn main() {
config := BenchConfig{
case_name: os.getenv('DTM2_BENCH_CASE')
iterations: env_int('DTM2_BENCH_ITERATIONS', 50000)
cold_iterations: env_int('DTM2_BENCH_COLD_ITERATIONS', 500)
placeholder_count: env_int('DTM2_BENCH_PLACEHOLDERS', 50)
compress_html: env_bool('DTM2_BENCH_COMPRESS_HTML', true)
reload_modified_templates: env_bool('DTM2_BENCH_RELOAD_MODIFIED_TEMPLATES', false)
validate_each_iteration: env_bool('DTM2_BENCH_VALIDATE_EACH_ITERATION', false)
}
root := os.join_path(os.vtmp_dir(), '${bench_root_name}_${os.getpid()}')
os.rmdir_all(root) or {}
setup_files(root, config)!
defer {
os.rmdir_all(root) or {}
}
print_config(root, config)
small := small_placeholders()
many := many_placeholders(config.placeholder_count)
include := include_placeholders()
if should_run(config, 'small_hot') {
bench_hot('small_hot', root, 'small.html', small, config.iterations, config)
}
if should_run(config, 'small_cold') {
bench_cold('small_cold', root, 'small.html', small, config.cold_iterations, config)
}
if should_run(config, 'many_hot') {
bench_hot('many_hot', root, 'many.html', many, config.iterations, config)
}
if should_run(config, 'many_cold') {
bench_cold('many_cold', root, 'many.html', many, config.cold_iterations, config)
}
if should_run(config, 'include_hot') {
bench_hot('include_hot', root, 'with_include.html', include, config.iterations, config)
}
if should_run(config, 'include_cold') {
bench_cold('include_cold', root, 'with_include.html', include, config.cold_iterations,
config)
}
if should_run(config, 'xml_hot') {
bench_hot('xml_hot', root, 'feed.xml', small, config.iterations, config)
}
if should_run(config, 'xml_cold') {
bench_cold('xml_cold', root, 'feed.xml', small, config.cold_iterations, config)
}
}
fn print_config(root string, config BenchConfig) {
println('DTM2 benchmark')
println('root: ${root}')
println('case_name: ${if config.case_name == '' { 'all' } else { config.case_name }}')
println('iterations: ${config.iterations}')
println('cold_iterations: ${config.cold_iterations}')
println('placeholder_count: ${config.placeholder_count}')
println('compress_html: ${config.compress_html}')
println('reload_modified_templates: ${config.reload_modified_templates}')
println('validate_each_iteration: ${config.validate_each_iteration}')
println('')
}
fn env_int(name string, default_value int) int {
raw := os.getenv(name)
if raw == '' {
return default_value
}
value := raw.int()
if value <= 0 {
return default_value
}
return value
}
fn env_bool(name string, default_value bool) bool {
raw := os.getenv(name).to_lower()
if raw == '' {
return default_value
}
if raw in ['1', 'true', 'yes', 'on'] {
return true
}
if raw in ['0', 'false', 'no', 'off'] {
return false
}
return default_value
}
fn should_run(config BenchConfig, label string) bool {
return config.case_name == '' || config.case_name == 'all' || config.case_name == label
}
fn setup_files(root string, config BenchConfig) ! {
os.mkdir_all(os.join_path(root, 'partials'))!
os.write_file(os.join_path(root, 'small.html'),
'<main>@title <p>@body</p><span>@count</span></main>')!
mut many := strings.new_builder(config.placeholder_count * 30)
many.writeln('<dl>')
for i := 0; i < config.placeholder_count; i++ {
many.writeln('<dt>${i}</dt><dd>@p_${i}</dd>')
}
many.writeln('</dl>')
os.write_file(os.join_path(root, 'many.html'), many.str())!
os.write_file(os.join_path(root, 'partials', 'nav.html'), '<nav>@title</nav>')!
os.write_file(os.join_path(root, 'with_include.html'),
'<header>@include "partials/nav"</header><main>@body</main>')!
os.write_file(os.join_path(root, 'feed.xml'),
'<feed><title>@title</title><entry>@body</entry><count>@count</count></feed>')!
}
fn small_placeholders() map[string]string {
mut placeholders := map[string]string{}
placeholders['title'] = 'Small'.clone()
placeholders['body'] = 'Body <escaped>'.clone()
placeholders['count'] = '42'
return placeholders
}
fn many_placeholders(count int) map[string]string {
mut placeholders := map[string]string{}
for i := 0; i < count; i++ {
placeholders['p_${i}'] = 'value ${i}'.clone()
}
return placeholders
}
fn include_placeholders() map[string]string {
mut placeholders := map[string]string{}
placeholders['title'] = 'Menu'.clone()
placeholders['body'] = 'Body <escaped>'.clone()
return placeholders
}
fn bench_hot(label string, root string, template_path string, placeholders map[string]string, iterations int, config BenchConfig) {
mut manager := new_manager(root, config)
expected := manager.expand(template_path, placeholders: &placeholders).clone()
mut last_len := expected.len
sw := time.new_stopwatch()
for i in 0 .. iterations {
rendered := manager.expand(template_path, placeholders: &placeholders)
if config.validate_each_iteration {
validate_rendered(label, i, expected, rendered)
}
last_len = rendered.len
}
elapsed := sw.elapsed()
validate_rendered(label, iterations, expected, manager.expand(template_path,
placeholders: &placeholders
))
print_result(label, iterations, elapsed, last_len, manager.compiled_template_count())
}
fn bench_cold(label string, root string, template_path string, placeholders map[string]string, iterations int, config BenchConfig) {
mut expected_manager := new_manager(root, config)
expected := expected_manager.expand(template_path, placeholders: &placeholders).clone()
mut last_len := expected.len
sw := time.new_stopwatch()
for i in 0 .. iterations {
mut manager := new_manager(root, config)
rendered := manager.expand(template_path, placeholders: &placeholders)
if config.validate_each_iteration {
validate_rendered(label, i, expected, rendered)
}
last_len = rendered.len
}
elapsed := sw.elapsed()
mut final_manager := new_manager(root, config)
validate_rendered(label, iterations, expected, final_manager.expand(template_path,
placeholders: &placeholders
))
print_result(label, iterations, elapsed, last_len, final_manager.compiled_template_count())
}
fn new_manager(root string, config BenchConfig) &dtm2.Manager {
return dtm2.initialize(
template_dir: root
compress_html: config.compress_html
reload_modified_templates: config.reload_modified_templates
)
}
fn validate_rendered(label string, iteration int, expected string, rendered string) {
if rendered != expected {
eprintln('${label} invalid output at iteration ${iteration}: expected_len=${expected.len} actual_len=${rendered.len}')
exit(1)
}
}
fn print_result(label string, iterations int, elapsed time.Duration, last_len int, compiled_count int) {
ns_per_op := if iterations > 0 { elapsed.nanoseconds() / i64(iterations) } else { i64(0) }
ops_per_sec := if ns_per_op > 0 { 1_000_000_000.0 / f64(ns_per_op) } else { 0.0 }
println('${label:-16} iterations=${iterations:8} total_ms=${elapsed.milliseconds():8} ns_per_op=${ns_per_op:10} ops_per_sec=${ops_per_sec:10.1f} last_len=${last_len:6} compiled_templates=${compiled_count}')
}
@@ -0,0 +1,57 @@
#!/usr/bin/env bash
set -euo pipefail
script_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
repo_root="$(cd "${script_dir}/../../../../.." && pwd)"
cd "${repo_root}"
timestamp="$(date +%Y-%m-%d_%H%M%S)"
results_dir="${DTM2_BENCH_RESULTS_DIR:-vlib/x/templating/dtm2/benchmarks/results/${timestamp}}"
bench_source="vlib/x/templating/dtm2/benchmarks/dtm2_benchmark.v"
bench_bin="/tmp/dtm2_benchmark_${timestamp}_$$"
bench_mode="${DTM2_BENCH_MODE:-prod}"
mkdir -p "${results_dir}"
trap 'rm -f "${bench_bin}"' EXIT
case "${bench_mode}" in
dev)
build_cmd=(./v -o "${bench_bin}" "${bench_source}")
;;
prod)
build_cmd=(./v -prod -o "${bench_bin}" "${bench_source}")
;;
prod_o2)
build_cmd=(./v -prod -no-prod-options -cflags -O2 -o "${bench_bin}" "${bench_source}")
;;
*)
echo "unsupported DTM2_BENCH_MODE=${bench_mode}; use dev, prod, or prod_o2" >&2
exit 2
;;
esac
{
echo "date: $(date -Iseconds)"
echo "v_version: $(./v --version)"
echo "kernel: $(uname -srmo)"
echo "cpu_model: $(awk -F': ' '/model name/{print $2; exit}' /proc/cpuinfo)"
echo "logical_cpu_count: $(nproc)"
echo "memory_total: $(awk '/MemTotal/{print $2 " " $3}' /proc/meminfo)"
echo "benchmark_mode: ${bench_mode}"
echo "benchmark_source: ${bench_source}"
echo "DTM2_BENCH_CASE: ${DTM2_BENCH_CASE:-all}"
echo "DTM2_BENCH_ITERATIONS: ${DTM2_BENCH_ITERATIONS:-50000}"
echo "DTM2_BENCH_COLD_ITERATIONS: ${DTM2_BENCH_COLD_ITERATIONS:-500}"
echo "DTM2_BENCH_PLACEHOLDERS: ${DTM2_BENCH_PLACEHOLDERS:-50}"
echo "DTM2_BENCH_COMPRESS_HTML: ${DTM2_BENCH_COMPRESS_HTML:-true}"
echo "DTM2_BENCH_RELOAD_MODIFIED_TEMPLATES: ${DTM2_BENCH_RELOAD_MODIFIED_TEMPLATES:-false}"
echo "DTM2_BENCH_VALIDATE_EACH_ITERATION: ${DTM2_BENCH_VALIDATE_EACH_ITERATION:-false}"
} > "${results_dir}/environment.txt"
"${build_cmd[@]}" > "${results_dir}/build.log" 2>&1
{
cat "${results_dir}/environment.txt"
echo
/usr/bin/time -v "${bench_bin}"
} 2>&1 | tee "${results_dir}/benchmark.log"
@@ -0,0 +1,4 @@
{
"html": [".html", ".htm", ".xml"],
"text": [".txt", ".text"]
}
@@ -0,0 +1,821 @@
module dtm2
import hash
import json
import os
import strings
// dtm2 is the modern runtime renderer for Dynamic Template Manager.
// It keeps DTM dynamic, meaning templates can still be edited on disk without
// recompiling the application, while moving the hot path to a parsed-template
// cache instead of a rendered-output cache. File extensions are configurable,
// but every template ultimately renders in either HTML mode or text mode.
//
// The intended runtime model is one long-lived Manager per application or
// rendering context. Tests and benchmarks may create short-lived managers, but
// normal applications should reuse a manager so parsed templates and resolved
// paths stay hot.
const message_signature = '[Dynamic Template Manager 2]'
const internal_server_error = 'Internal Server Error'
const include_html_key_suffix = '_#includehtml'
const include_directive = '@include '
const max_include_depth = 32
const max_extension_config_size = 64 * 1024
const default_extension_config_filename = 'dtm2_extensions.json'
const segment_instruction_size = 9
const segment_kind_text = u8(0)
const segment_kind_placeholder = u8(1)
// These tags are the compatibility allow-list for the historical `_#includehtml`
// placeholder suffix. Any other tag is still escaped.
const allowed_html_tags = ['<div>', '</div>', '<h1>', '</h1>', '<h2>', '</h2>', '<h3>', '</h3>',
'<h4>', '</h4>', '<h5>', '</h5>', '<h6>', '</h6>', '<p>', '</p>', '<br>', '<hr>', '<span>',
'</span>', '<ul>', '</ul>', '<ol>', '</ol>', '<li>', '</li>', '<dl>', '</dl>', '<dt>', '</dt>',
'<dd>', '</dd>', '<menu>', '</menu>', '<table>', '</table>', '<caption>', '</caption>', '<th>',
'</th>', '<tr>', '</tr>', '<td>', '</td>', '<thead>', '</thead>', '<thread>', '</thread>',
'<tbody>', '</tbody>', '<tfoot>', '</tfoot>', '<col>', '</col>', '<colgroup>', '</colgroup>',
'<header>', '</header>', '<footer>', '</footer>', '<main>', '</main>', '<section>', '</section>',
'<article>', '</article>', '<aside>', '</aside>', '<details>', '</details>', '<dialog>',
'</dialog>', '<data>', '</data>', '<summary>', '</summary>']!
pub enum TemplateType {
html
text
}
// ExtensionConfig is the JSON representation accepted by
// `ManagerParams.extension_config_file`.
//
// Example:
//
// ```json
// {
// "html": [".html", ".htm", ".xml", ".view"],
// "text": [".txt", ".mail"]
// }
// ```
pub struct ExtensionConfig {
pub:
html []string
text []string
}
// TemplateDependency records the file metadata that decides whether a parsed
// template is still fresh. Includes are tracked as dependencies of the parent
// template, so editing a partial can invalidate the compiled parent tree.
struct TemplateDependency {
path string
modified_at i64
size u64
content_hash u64
}
// CompiledTemplate is the parsed representation stored by Manager.
//
// It intentionally stores a compact instruction string instead of slices of
// segment structs. The instructions contain offsets into the owned `content`
// string, which keeps the cache compact and avoids keeping fragile string-slice
// graphs alive across many -prod renders.
@[heap]
struct CompiledTemplate {
// Rendering mode inferred from the source file extension.
template_type TemplateType
// Full template content after static @include expansion.
content string
// Compact metadata used to decide whether this compiled tree is stale.
dependency_signature string
// Binary instruction stream: one fixed-size record per text/placeholder segment.
instructions string
// Number of text and placeholder records in `instructions`.
segment_count int
// Static-text size hint used to preallocate the render buffer.
estimated_size int
}
// Manager owns the runtime state of a dtm2 renderer.
//
// It caches resolved template paths and parsed template trees, but never caches
// rendered HTML responses. This keeps the new engine deterministic and leaves
// legacy rendered-cache compatibility in `x.templating.dtm`.
@[heap]
pub struct Manager {
mut:
// Base directory for relative template paths.
template_dir string
// Enables the lightweight deterministic HTML whitespace compressor.
compress_html bool
// When true, source and include files are stat-checked before cache reuse.
reload_modified_templates bool
// Maps caller-provided template paths to canonical source paths.
resolved_template_paths map[string]string
// Parsed-template cache keyed by canonical source path.
compiled_templates map[string]&CompiledTemplate
// User-provided extension-to-render-mode overrides. Built-in extensions are
// resolved without allocating a per-manager default map.
template_extensions map[string]TemplateType
}
// ManagerParams configure a dtm2 Manager.
@[params]
pub struct ManagerParams {
pub:
// Root directory used when `expand()` receives a relative template path.
// If empty, `<executable directory>/templates` is used.
template_dir string
// Compresses HTML output by removing newlines/tabs and redundant spaces.
compress_html bool = true
// Re-check source files and includes before reusing a parsed template tree.
// Disable it for maximum hot-path throughput when templates are immutable.
reload_modified_templates bool = true
// Additional or overriding extension mappings.
// Default mappings are: `.html`, `.htm`, `.xml` => HTML mode and
// `.txt`, `.text` => text mode.
template_extensions map[string]TemplateType
// Optional JSON file containing extension mappings. It is merged after the
// default mappings and before `template_extensions`, so explicit code
// configuration wins over the file. If empty, DTM2 tries to load
// `<template_dir>/dtm2_extensions.json` when that file exists.
extension_config_file string
}
// RenderParams configure one render call.
@[params]
pub struct RenderParams {
pub:
// Placeholder values keyed by their template name without the `@` prefix.
// Values are escaped by default.
placeholders &map[string]string = &map[string]string{}
// Prefix written back when a placeholder is missing. The default preserves
// the original `@placeholder` text.
missing_placeholder_prefix string = '@'
}
// initialize creates a dynamic template manager rooted at `params.template_dir`.
//
// The returned manager should normally be kept and reused. Reusing it is what
// gives dtm2 its parsed-template and path-resolution cache benefits.
pub fn initialize(params ManagerParams) &Manager {
raw_template_dir := if params.template_dir == '' {
os.join_path(os.dir(os.executable()), 'templates')
} else {
params.template_dir
}
template_dir := canonical_template_dir(raw_template_dir)
template_extensions := build_template_extensions(params, template_dir)
return &Manager{
template_dir: template_dir.clone()
compress_html: params.compress_html
reload_modified_templates: params.reload_modified_templates
resolved_template_paths: map[string]string{}
compiled_templates: map[string]&CompiledTemplate{}
template_extensions: template_extensions
}
}
fn canonical_template_dir(template_dir string) string {
return os.real_path(template_dir)
}
fn build_template_extensions(params ManagerParams, template_dir string) map[string]TemplateType {
mut extensions := map[string]TemplateType{}
config_path := extension_config_path(params, template_dir)
if config_path != '' {
file_extensions := read_extension_config_file(config_path) or {
eprintln('${message_signature} ${err.msg()}')
map[string]TemplateType{}
}
merge_template_extensions(mut extensions, file_extensions)
}
merge_template_extensions(mut extensions, params.template_extensions)
return extensions
}
fn extension_config_path(params ManagerParams, template_dir string) string {
if params.extension_config_file != '' {
return params.extension_config_file
}
default_path := os.join_path(template_dir, default_extension_config_filename)
if os.exists(default_path) {
return default_path
}
return ''
}
fn read_extension_config_file(config_path string) !map[string]TemplateType {
if !os.exists(config_path) {
return error('extension config file "${config_path}" not found')
}
if os.is_dir(config_path) {
return error('extension config path "${config_path}" is a directory')
}
config_stat := os.stat(config_path)!
if config_stat.size > u64(max_extension_config_size) {
return error('extension config file "${config_path}" is larger than ${max_extension_config_size} bytes')
}
raw_config := os.read_file(config_path)!
config := json.decode(ExtensionConfig, raw_config)!
mut extensions := map[string]TemplateType{}
merge_template_extension_list(mut extensions, config.html, .html)
merge_template_extension_list(mut extensions, config.text, .text)
return extensions
}
fn merge_template_extensions(mut target map[string]TemplateType, source map[string]TemplateType) {
for ext, template_type in source {
normalized := validate_template_extension(ext) or {
eprintln('${message_signature} ${err.msg()}')
continue
}
target[normalized] = template_type
}
}
fn merge_template_extension_list(mut target map[string]TemplateType, extensions []string, template_type TemplateType) {
for ext in extensions {
normalized := validate_template_extension(ext) or {
eprintln('${message_signature} ${err.msg()}')
continue
}
target[normalized] = template_type
}
}
fn normalize_template_extension(ext string) string {
trimmed := ext.trim_space().to_lower()
if trimmed == '' {
return ''
}
if trimmed.starts_with('.') {
return trimmed
}
return '.${trimmed}'
}
fn validate_template_extension(ext string) !string {
normalized := normalize_template_extension(ext)
if normalized.len < 2 {
return error('ignoring invalid template extension "${ext}"')
}
for i := 1; i < normalized.len; i++ {
c := normalized[i]
if (c >= `a` && c <= `z`) || (c >= `0` && c <= `9`) || c == `_` || c == `-` || c == `.` {
continue
}
return error('ignoring invalid template extension "${ext}"')
}
return normalized
}
// compiled_template_count is intentionally exposed for tests and diagnostics. In
// dtm2 it counts parsed template trees currently held by the manager.
pub fn (m &Manager) compiled_template_count() int {
return m.compiled_templates.len
}
// expand renders `template_path` with the provided placeholders.
//
// `template_path` can be absolute or relative to the manager's template
// directory. Supported extensions come from the manager extension table.
// Errors are reported to stderr and return the legacy-compatible
// `Internal Server Error` string.
pub fn (mut m Manager) expand(template_path string, params RenderParams) string {
source_path := m.cached_template_path(template_path) or {
eprintln('${message_signature} ${err.msg()}')
return internal_server_error
}
compiled := m.compiled_template_for_path(source_path) or {
eprintln('${message_signature} ${err.msg()}')
return internal_server_error
}
if m.compress_html && compiled.template_type == .html {
rendered := render_compiled_template(compiled, params)
compressed := compress_html_output(rendered)
return compressed.clone()
}
rendered := render_compiled_template(compiled, params)
return rendered.clone()
}
// cached_template_path reuses canonical paths when reload checks are disabled.
// When reload checks are enabled, it revalidates the current real path so a
// replaced symlink cannot bypass the template directory boundary.
fn (mut m Manager) cached_template_path(template_path string) !string {
if source_path := m.resolved_template_paths[template_path] {
if m.reload_modified_templates {
current_source_path := m.resolve_template_path(template_path)!
if current_source_path != source_path {
m.resolved_template_paths[template_path] = current_source_path
return current_source_path
}
}
return source_path
}
source_path := m.resolve_template_path(template_path)!
m.resolved_template_paths[template_path] = source_path
return source_path
}
// compiled_template_for_path returns the parsed tree for a canonical source
// path. When reload checks are enabled, dependency metadata decides whether the
// tree can be reused or must be rebuilt from disk.
fn (mut m Manager) compiled_template_for_path(source_path string) !&CompiledTemplate {
if compiled := m.compiled_templates[source_path] {
if !m.reload_modified_templates || compiled.dependencies_are_fresh(m.template_dir) {
return compiled
}
}
compiled := compile_template_from_file(source_path, m.template_dir, m.template_extensions,
m.reload_modified_templates)!
m.compiled_templates[source_path] = compiled
return compiled
}
// compile_template_from_file reads a template, expands static includes, parses
// placeholders, and stores dependency metadata for future invalidation.
fn compile_template_from_file(source_path string, template_root string, template_extensions map[string]TemplateType, track_dependencies bool) !&CompiledTemplate {
canonical_path := os.real_path(source_path)
ensure_path_inside_template_dir(canonical_path, template_root)!
template_type := template_type_from_path(canonical_path, template_extensions)!
content, dependencies := read_template_with_includes(canonical_path, template_root, 0,
track_dependencies)!
instructions, estimated_size, segment_count := parse_segments(content)
dependency_signature := encode_dependencies(dependencies)
return &CompiledTemplate{
template_type: template_type
content: copy_string(content)
dependency_signature: copy_string(dependency_signature)
instructions: copy_string(instructions)
segment_count: segment_count
estimated_size: estimated_size
}
}
// copy_string forces cached strings to own their memory. Cached templates live
// beyond the stack frame that built them, so the cache should not retain
// accidental views into temporary buffers.
fn copy_string(value string) string {
mut bytes := []u8{len: value.len}
for i := 0; i < value.len; i++ {
bytes[i] = value[i]
}
copied := bytes.bytestr()
return copied.clone()
}
// dependencies_are_fresh checks root and included files. The content hash closes
// the same-second, same-size edit window left by second-resolution mtimes.
fn (compiled &CompiledTemplate) dependencies_are_fresh(template_root string) bool {
signature := compiled.dependency_signature
mut offset := 0
for offset < signature.len {
first_sep := index_byte_from(signature, `|`, offset) or { return false }
second_sep := index_byte_from(signature, `|`, first_sep + 1) or { return false }
third_sep := index_byte_from(signature, `|`, second_sep + 1) or { return false }
line_end := index_byte_from(signature, `\n`, third_sep + 1) or { signature.len }
size := signature[first_sep + 1..second_sep].i64()
if size < 0 {
return false
}
content_hash := signature[second_sep + 1..third_sep].u64()
path := signature[third_sep + 1..line_end]
current_path := os.real_path(path)
ensure_path_inside_template_dir(current_path, template_root) or { return false }
if current_path != path {
return false
}
stat := os.stat(path) or { return false }
if stat.mtime != signature[offset..first_sep].i64() || stat.size != u64(size) {
return false
}
content := os.read_file(path) or { return false }
if content_fingerprint(content) != content_hash {
return false
}
offset = line_end + 1
}
return true
}
fn index_byte_from(value string, needle u8, start int) ?int {
for i := start; i < value.len; i++ {
if value[i] == needle {
return i
}
}
return none
}
fn encode_dependencies(dependencies []TemplateDependency) string {
mut out := strings.new_builder(dependencies.len * 64)
for dependency in dependencies {
out.writeln('${dependency.modified_at}|${dependency.size}|${dependency.content_hash}|${dependency.path}')
}
encoded := out.str()
return encoded.clone()
}
// resolve_template_path converts absolute or manager-relative template names to
// canonical paths. The canonical path is used as the compiled-template cache key.
fn (m &Manager) resolve_template_path(template_path string) !string {
source_path := if os.is_abs_path(template_path) {
template_path
} else {
os.join_path(m.template_dir, template_path)
}
if !os.exists(source_path) {
return error('template "${source_path}" not found')
}
canonical_path := os.real_path(source_path)
ensure_path_inside_template_dir(canonical_path, m.template_dir)!
return canonical_path
}
fn ensure_path_inside_template_dir(path string, template_root string) ! {
root := path_without_trailing_separator(template_root)
candidate := path_without_trailing_separator(path)
if candidate == root || candidate.starts_with(root + os.path_separator) {
return
}
return error('template "${path}" is outside template directory "${template_root}"')
}
fn path_without_trailing_separator(path string) string {
if path.len > os.path_separator.len && path.ends_with(os.path_separator) {
return path[..path.len - os.path_separator.len]
}
return path
}
// template_type_from_path keeps rendering rules extension-driven and explicit.
fn template_type_from_path(source_path string, template_extensions map[string]TemplateType) !TemplateType {
ext := normalize_template_extension(os.file_ext(source_path))
if template_extensions.len > 0 {
if template_type := template_extensions[ext] {
return template_type
}
}
if template_type := default_template_type_from_extension(ext) {
return template_type
}
return error('template "${source_path}" uses unsupported extension "${ext}"')
}
fn default_template_type_from_extension(ext string) ?TemplateType {
match ext {
'.html', '.htm', '.xml' {
return .html
}
'.txt', '.text' {
return .text
}
else {
return none
}
}
}
// read_template_with_includes expands simple line-level `@include "path"`
// directives before parsing placeholders. Includes are part of the parsed tree
// and are tracked as dependencies for reload checks.
fn read_template_with_includes(source_path string, template_root string, depth int, track_dependencies bool) !(string, []TemplateDependency) {
if depth > max_include_depth {
return error('maximum @include depth exceeded while reading "${source_path}"')
}
content := os.read_file(source_path)!
mut dependencies := []TemplateDependency{cap: 4}
if track_dependencies {
dependencies << template_dependency(source_path, content)!
}
if !content_needs_include_expansion(content) {
return content_with_template_newline(content), dependencies
}
base_dir := os.dir(source_path)
mut out := strings.new_builder(content.len)
lines := content.split_into_lines()
for line in lines {
expanded_line, line_dependencies := expand_include_directives(line, base_dir,
template_root, depth, track_dependencies)!
out.write_string(expanded_line)
out.write_u8(`\n`)
dependencies << line_dependencies
}
rendered := out.str()
return rendered.clone(), dependencies
}
fn content_with_template_newline(content string) string {
if content == '' || content[content.len - 1] == `\n` {
return content.clone()
}
with_newline := content + '\n'
return with_newline.clone()
}
fn content_needs_include_expansion(content string) bool {
for i := 0; i < content.len; i++ {
if content[i] == `\r` {
return true
}
if is_include_directive_at(content, i) {
return true
}
}
return false
}
fn is_include_directive_at(content string, pos int) bool {
return pos + include_directive.len <= content.len && content[pos] == `@`
&& content[pos + 1] == `i` && content[pos + 2] == `n` && content[pos + 3] == `c`
&& content[pos + 4] == `l` && content[pos + 5] == `u` && content[pos + 6] == `d`
&& content[pos + 7] == `e` && content[pos + 8] == ` `
}
fn template_dependency(source_path string, content string) !TemplateDependency {
stat := os.stat(source_path)!
return TemplateDependency{
path: source_path.clone()
modified_at: stat.mtime
size: stat.size
content_hash: content_fingerprint(content)
}
}
fn content_fingerprint(content string) u64 {
return hash.sum64_string(content, 0)
}
fn expand_include_directives(line string, base_dir string, template_root string, depth int, track_dependencies bool) !(string, []TemplateDependency) {
mut out := strings.new_builder(line.len)
mut dependencies := []TemplateDependency{}
mut offset := 0
for {
rel_pos := line[offset..].index(include_directive) or {
out.write_string(line[offset..])
break
}
pos := offset + rel_pos
target, end_pos := include_target_from_line_at(line, pos) or {
out.write_string(line[offset..pos + include_directive.len])
offset = pos + include_directive.len
continue
}
out.write_string(line[offset..pos])
include_path := resolve_include_path(base_dir, target, template_root)!
next_depth := depth + 1
included, include_dependencies := read_template_with_includes(include_path, template_root,
next_depth, track_dependencies)!
out.write_string(included.trim_right('\n'))
dependencies << include_dependencies
offset = end_pos
}
expanded := out.str()
return expanded.clone(), dependencies
}
fn include_target_from_line_at(line string, pos int) ?(string, int) {
mut cursor := pos + include_directive.len
for cursor < line.len && line[cursor].is_space() {
cursor++
}
if cursor >= line.len {
return none
}
quote := line[cursor]
if quote != `'` && quote != `"` {
return none
}
start := cursor + 1
for cursor = start; cursor < line.len; cursor++ {
if line[cursor] == quote {
return line[start..cursor], cursor + 1
}
}
return none
}
fn resolve_include_path(base_dir string, include_target string, template_root string) !string {
mut target := include_target
if os.file_ext(target) == '' {
target += '.html'
}
source_path := if os.is_abs_path(target) {
target
} else {
os.join_path(base_dir, target)
}
if !os.exists(source_path) {
return error('included template "${source_path}" not found')
}
include_path := os.real_path(source_path)
ensure_path_inside_template_dir(include_path, template_root)!
return include_path
}
// parse_segments converts template content to a compact instruction stream.
// Each instruction stores a segment kind plus offset/length into `content`.
// Rendering can then walk the stream without reparsing placeholder syntax.
fn parse_segments(content string) (string, int, int) {
mut instructions := []u8{cap: segment_instruction_size * 16}
mut text_start := 0
mut estimated_size := 0
mut segment_count := 0
mut i := 0
for i < content.len {
if content[i] != `@` || i + 1 >= content.len || !is_placeholder_char(content[i + 1]) {
i++
continue
}
if i > text_start {
append_segment_instruction(mut instructions, segment_kind_text, text_start,
i - text_start)
segment_count++
estimated_size += i - text_start
}
mut end := i + 1
for end < content.len && is_placeholder_char(content[end]) {
end++
}
append_segment_instruction(mut instructions, segment_kind_placeholder, i + 1, end - i - 1)
segment_count++
i = end
text_start = end
}
if text_start < content.len {
append_segment_instruction(mut instructions, segment_kind_text, text_start,
content.len - text_start)
segment_count++
estimated_size += content.len - text_start
}
encoded := instructions.bytestr()
return encoded.clone(), estimated_size, segment_count
}
fn append_segment_instruction(mut instructions []u8, kind u8, start int, len int) {
instructions << kind
append_u32(mut instructions, start)
append_u32(mut instructions, len)
}
fn append_u32(mut data []u8, value int) {
encoded := u32(value)
data << u8(encoded & 0xff)
data << u8((encoded >> 8) & 0xff)
data << u8((encoded >> 16) & 0xff)
data << u8((encoded >> 24) & 0xff)
}
fn read_u32(data string, offset int) int {
b0 := u32(data[offset])
b1 := u32(data[offset + 1]) << 8
b2 := u32(data[offset + 2]) << 16
b3 := u32(data[offset + 3]) << 24
value := b0 | b1 | b2 | b3
return int(value)
}
fn is_placeholder_char(c u8) bool {
return (c >= `a` && c <= `z`) || (c >= `A` && c <= `Z`) || (c >= `0` && c <= `9`) || c == `_`
}
// render_compiled_template walks the parsed instruction stream and writes the
// final output. Placeholder values are rendered only for this call; the manager
// never stores caller data after `expand()` returns.
fn render_compiled_template(compiled &CompiledTemplate, params RenderParams) string {
mut out := strings.new_builder(estimate_render_size(compiled))
mut offset := 0
for offset + segment_instruction_size <= compiled.instructions.len {
kind := compiled.instructions[offset]
start := read_u32(compiled.instructions, offset + 1)
len := read_u32(compiled.instructions, offset + 5)
offset += segment_instruction_size
if kind == segment_kind_text {
text := compiled.segment_text(start, len)
out.write_string(text)
continue
}
if kind == segment_kind_placeholder {
name := compiled.segment_text(start, len)
if write_placeholder_value(mut out, name, params.placeholders, compiled.template_type) {
continue
}
out.write_string(params.missing_placeholder_prefix)
out.write_string(name)
}
}
rendered := out.str()
return rendered.clone()
}
fn (compiled &CompiledTemplate) segment_text(start int, len int) string {
end := start + len
if start < 0 || len < 0 || end > compiled.content.len {
return ''
}
return compiled.content[start..end]
}
// write_placeholder_value resolves the normal placeholder name first, then the
// historical `_#includehtml` alias. The alias is accepted for compatibility but
// still passes through the allow-list based HTML escaping path.
fn write_placeholder_value(mut out strings.Builder, name string, placeholders &map[string]string, template_type TemplateType) bool {
unsafe {
if raw_value := placeholders[name] {
rendered_value := render_value(raw_value, name.ends_with(include_html_key_suffix),
template_type)
out.write_string(rendered_value)
return true
}
include_html_name := name + include_html_key_suffix
if raw_html := placeholders[include_html_name] {
rendered_html := render_value(raw_html, true, template_type)
out.write_string(rendered_html)
return true
}
}
return false
}
// estimate_render_size gives strings.Builder enough capacity for common
// placeholder expansion without making a second pre-render pass.
fn estimate_render_size(compiled &CompiledTemplate) int {
return compiled.estimated_size + 1024 + (compiled.segment_count * 64)
}
// render_value applies DTM's safety default: values are escaped unless the
// caller explicitly uses the include-html convention in an HTML template.
fn render_value(raw string, allow_html bool, template_type TemplateType) string {
if allow_html && template_type == .html {
return escape_with_allowed_html(raw)
}
return escape_html(raw)
}
// escape_with_allowed_html escapes the full value first, then restores only the
// supported tags. This preserves the legacy opt-in behavior without allowing
// arbitrary raw HTML through.
fn escape_with_allowed_html(value string) string {
mut escaped := escape_html(value)
for tag in allowed_html_tags {
escaped = escaped.replace(escape_html(tag), tag)
}
return escaped
}
// escape_html is a local deterministic HTML escape helper. It avoids depending
// on heavier generic replacement paths in the hot render loop.
fn escape_html(value string) string {
mut escaped := strings.new_builder(value.len)
for i := 0; i < value.len; i++ {
match value[i] {
`&` {
escaped.write_string('&amp;')
}
`<` {
escaped.write_string('&lt;')
}
`>` {
escaped.write_string('&gt;')
}
`"` {
escaped.write_string('&#34;')
}
`'` {
escaped.write_string('&#39;')
}
else {
escaped.write_u8(value[i])
}
}
}
escaped_value := escaped.str()
return escaped_value.clone()
}
// compress_html_output performs a small, predictable whitespace compression
// pass for HTML templates. It intentionally avoids regex work in the hot path.
fn compress_html_output(html string) string {
mut result := strings.new_builder(html.len)
mut pending_space := false
mut last_written := u8(0)
for i := 0; i < html.len; i++ {
c := html[i]
if c == `\n` || c == `\t` {
continue
}
if c == ` ` {
pending_space = true
continue
}
if pending_space {
if !(last_written == `>` && c == `<`) {
result.write_u8(` `)
last_written = ` `
}
pending_space = false
}
result.write_u8(c)
last_written = c
}
compressed := result.str()
return compressed.clone()
}
@@ -0,0 +1,429 @@
module dtm2
import os
import time
const test_root_name = 'dtm2_test'
fn dtm2_test_root() string {
return os.join_path(os.vtmp_dir(), '${test_root_name}_${os.getpid()}')
}
fn dtm2_outside_template_path() string {
return os.join_path(os.vtmp_dir(), '${test_root_name}_${os.getpid()}_outside.html')
}
fn testsuite_begin() {
root := dtm2_test_root()
os.rmdir_all(root) or {}
os.mkdir_all(os.join_path(root, 'partials'))!
outside_path := dtm2_outside_template_path()
os.rm(outside_path) or {}
os.write_file(outside_path, '<p>@title outside</p>')!
os.write_file(os.join_path(root, 'page.html'), '<main>@title @body @missing</main>')!
os.write_file(os.join_path(root, 'invalid.css'), 'body { color: red; }')!
os.write_file(os.join_path(root, 'raw.txt'), 'Raw: @body')!
os.write_file(os.join_path(root, 'with_include.html'),
'<header>@include "partials/nav"</header><main>@title</main>')!
os.write_file(os.join_path(root, 'partials', 'nav.html'), '<nav>@title</nav>')!
os.write_file(os.join_path(root, 'include_outside_relative.html'),
'@include "../${os.base(outside_path)}"')!
os.write_file(os.join_path(root, 'include_outside_absolute.html'), '@include "${outside_path}"')!
os.write_file(os.join_path(root, 'reload.html'), '<p>@title</p>')!
os.write_file(os.join_path(root, 'pinned.html'), '<em>@title</em>')!
os.write_file(os.join_path(root, 'path_alias.html'), '<section>@title</section>')!
os.write_file(os.join_path(root, 'prefix.html'), '<p>@unknown</p>')!
os.write_file(os.join_path(root, 'recursive.html'), '@include "recursive"')!
os.write_file(os.join_path(root, 'compress.html'), '<div>\n <span>@title</span>\n</div>')!
os.write_file(os.join_path(root, 'empty.html'), '')!
os.write_file(os.join_path(root, 'feed.xml'), '<feed><title>@title</title></feed>')!
os.write_file(os.join_path(root, 'custom.page'), '<article>@title</article>')!
os.write_file(os.join_path(root, 'custom.note'), 'Note: @body')!
os.write_file(os.join_path(root, 'override.html'), '<div>\n @title\n</div>')!
os.write_file(os.join_path(root, 'custom.view'), '<section>@title</section>')!
os.write_file(os.join_path(root, 'custom.mail'), 'Subject: @title')!
os.write_file(os.join_path(root, 'auto.fragment'), '<aside>@title</aside>')!
os.write_file(os.join_path(root, 'auto.message'), 'Auto: @title')!
os.write_file(os.join_path(root, 'bad.path'), '<p>@title</p>')!
os.write_file(os.join_path(root, 'extensions.json'), '{"html":[".view"],"text":["mail"]}')!
os.write_file(os.join_path(root, default_extension_config_filename),
'{"html":[".fragment"],"text":[".message"]}')!
os.write_file(os.join_path(root, 'invalid_extensions.json'),
'{"html":["../bad"],"text":["bad/value",".message"]}')!
}
fn testsuite_end() {
os.rmdir_all(dtm2_test_root()) or {}
os.rm(dtm2_outside_template_path()) or {}
}
fn new_test_manager(compress_html bool) &Manager {
return initialize(
template_dir: dtm2_test_root()
compress_html: compress_html
)
}
fn test_render_placeholders_escape_and_preserve_missing() {
mut manager := new_test_manager(false)
placeholders := {
'title': 'Hello'
'body': '<script>alert(1)</script>'
}
rendered := manager.expand('page.html', placeholders: &placeholders)
assert rendered == '<main>Hello &lt;script&gt;alert(1)&lt;/script&gt; @missing</main>\n'
assert manager.compiled_template_count() == 1
}
fn test_include_and_compiled_template_registry() {
mut manager := new_test_manager(true)
placeholders := {
'title': 'Home'
}
rendered := manager.expand('with_include.html', placeholders: &placeholders)
assert rendered == '<header><nav>Home</nav></header><main>Home</main>'
assert manager.compiled_template_count() == 1
_ = manager.expand('with_include.html', placeholders: &placeholders)
assert manager.compiled_template_count() == 1
}
fn test_include_html_suffix_keeps_only_allowed_tags() {
mut manager := new_test_manager(false)
placeholders := {
'body_#includehtml': '<thead><tr><td>safe</td></tr></thead><script>bad</script>'
'title': 'HTML'
}
rendered := manager.expand('page.html', placeholders: &placeholders)
assert rendered.contains('<thead><tr><td>safe</td></tr></thead>')
assert rendered.contains('&lt;script&gt;bad&lt;/script&gt;')
}
fn test_text_mode_always_escapes_html() {
mut manager := new_test_manager(false)
placeholders := {
'body_#includehtml': '<span>text</span>'
}
rendered := manager.expand('raw.txt', placeholders: &placeholders)
assert rendered == 'Raw: &lt;span&gt;text&lt;/span&gt;\n'
}
fn test_reload_modified_template_without_render_cache() {
mut manager := new_test_manager(false)
placeholders := {
'title': 'One'
}
first := manager.expand('reload.html', placeholders: &placeholders)
assert first == '<p>One</p>\n'
os.write_file(os.join_path(dtm2_test_root(), 'reload.html'), '<strong>@title</strong>')!
second := manager.expand('reload.html', placeholders: &placeholders)
assert second == '<strong>One</strong>\n'
assert manager.compiled_template_count() == 1
}
fn test_reload_same_second_same_size_template_content_change() {
path := os.join_path(dtm2_test_root(), 'same_size.html')
fixed_time := i64(2_306_102_495)
os.write_file(path, '<p>@title A</p>')!
os.utime(path, fixed_time, fixed_time)!
mut manager := new_test_manager(false)
placeholders := {
'title': 'One'
}
first := manager.expand('same_size.html', placeholders: &placeholders)
assert first == '<p>One A</p>\n'
os.write_file(path, '<b>@title B</b>')!
os.utime(path, fixed_time, fixed_time)!
second := manager.expand('same_size.html', placeholders: &placeholders)
assert second == '<b>One B</b>\n'
assert manager.compiled_template_count() == 1
}
fn test_reload_same_second_same_size_include_content_change() {
parent_path := os.join_path(dtm2_test_root(), 'same_size_include_parent.html')
include_path := os.join_path(dtm2_test_root(), 'partials', 'same_size_include.html')
fixed_time := i64(2_306_102_496)
os.write_file(parent_path, '<main>@include "partials/same_size_include"</main>')!
os.write_file(include_path, '<p>@title A</p>')!
os.utime(include_path, fixed_time, fixed_time)!
mut manager := new_test_manager(false)
placeholders := {
'title': 'One'
}
first := manager.expand('same_size_include_parent.html', placeholders: &placeholders)
assert first == '<main><p>One A</p></main>\n'
os.write_file(include_path, '<b>@title B</b>')!
os.utime(include_path, fixed_time, fixed_time)!
second := manager.expand('same_size_include_parent.html', placeholders: &placeholders)
assert second == '<main><b>One B</b></main>\n'
assert manager.compiled_template_count() == 1
}
fn test_reload_modified_include_without_render_cache() {
mut manager := new_test_manager(true)
placeholders := {
'title': 'Home'
}
first := manager.expand('with_include.html', placeholders: &placeholders)
assert first == '<header><nav>Home</nav></header><main>Home</main>'
time.sleep(1100 * time.millisecond)
os.write_file(os.join_path(dtm2_test_root(), 'partials', 'nav.html'),
'<nav class="updated">@title</nav>')!
second := manager.expand('with_include.html', placeholders: &placeholders)
assert second == '<header><nav class="updated">Home</nav></header><main>Home</main>'
assert manager.compiled_template_count() == 1
}
fn test_reload_disabled_keeps_cached_template_tree() {
mut manager := initialize(
template_dir: dtm2_test_root()
compress_html: false
reload_modified_templates: false
)
placeholders := {
'title': 'Pinned'
}
first := manager.expand('pinned.html', placeholders: &placeholders)
assert first == '<em>Pinned</em>\n'
os.write_file(os.join_path(dtm2_test_root(), 'pinned.html'), '<strong>@title</strong>')!
second := manager.expand('pinned.html', placeholders: &placeholders)
assert second == '<em>Pinned</em>\n'
assert manager.compiled_template_count() == 1
}
fn test_relative_and_absolute_paths_share_compiled_template() {
mut manager := new_test_manager(false)
placeholders := {
'title': 'Alias'
}
relative := manager.expand('path_alias.html', placeholders: &placeholders)
absolute := manager.expand(os.join_path(dtm2_test_root(), 'path_alias.html'),
placeholders: &placeholders
)
assert relative == '<section>Alias</section>\n'
assert absolute == relative
assert manager.compiled_template_count() == 1
}
fn test_template_path_cannot_escape_template_dir() {
mut manager := new_test_manager(false)
placeholders := {
'title': 'Escape'
}
relative_escape := manager.expand('../${os.base(dtm2_outside_template_path())}',
placeholders: &placeholders
)
absolute_escape := manager.expand(dtm2_outside_template_path(), placeholders: &placeholders)
assert relative_escape == internal_server_error
assert absolute_escape == internal_server_error
assert manager.compiled_template_count() == 0
}
fn test_cached_template_path_revalidates_symlink_escape() {
$if windows {
return
}
path := os.join_path(dtm2_test_root(), 'symlink_swap.html')
os.write_file(path, '<p>@title safe</p>')!
mut manager := new_test_manager(false)
placeholders := {
'title': 'Link'
}
first := manager.expand('symlink_swap.html', placeholders: &placeholders)
assert first == '<p>Link safe</p>\n'
os.rm(path)!
os.symlink(dtm2_outside_template_path(), path) or { return }
escaped := manager.expand('symlink_swap.html', placeholders: &placeholders)
assert escaped == internal_server_error
}
fn test_cached_include_dependency_revalidates_symlink_escape() {
$if windows {
return
}
parent_path := os.join_path(dtm2_test_root(), 'include_symlink_swap_parent.html')
include_path := os.join_path(dtm2_test_root(), 'partials', 'symlink_swap.html')
os.write_file(parent_path, '<main>@include "partials/symlink_swap"</main>')!
os.write_file(include_path, '<p>@title safe</p>')!
mut manager := new_test_manager(false)
placeholders := {
'title': 'Link'
}
first := manager.expand('include_symlink_swap_parent.html', placeholders: &placeholders)
assert first == '<main><p>Link safe</p></main>\n'
os.rm(include_path)!
os.symlink(dtm2_outside_template_path(), include_path) or { return }
escaped := manager.expand('include_symlink_swap_parent.html', placeholders: &placeholders)
assert escaped == internal_server_error
}
fn test_reload_disabled_does_not_reopen_cached_symlink_swap() {
$if windows {
return
}
path := os.join_path(dtm2_test_root(), 'pinned_symlink_swap.html')
os.write_file(path, '<p>@title safe</p>')!
mut manager := initialize(
template_dir: dtm2_test_root()
compress_html: false
reload_modified_templates: false
)
placeholders := {
'title': 'Pinned'
}
first := manager.expand('pinned_symlink_swap.html', placeholders: &placeholders)
assert first == '<p>Pinned safe</p>\n'
os.rm(path)!
os.symlink(dtm2_outside_template_path(), path) or { return }
second := manager.expand('pinned_symlink_swap.html', placeholders: &placeholders)
assert second == first
assert manager.compiled_template_count() == 1
}
fn test_include_path_cannot_escape_template_dir() {
mut manager := new_test_manager(false)
placeholders := {
'title': 'Escape'
}
relative_include := manager.expand('include_outside_relative.html', placeholders: &placeholders)
absolute_include := manager.expand('include_outside_absolute.html', placeholders: &placeholders)
assert relative_include == internal_server_error
assert absolute_include == internal_server_error
assert manager.compiled_template_count() == 0
}
fn test_custom_missing_placeholder_prefix() {
mut manager := new_test_manager(false)
placeholders := map[string]string{}
rendered := manager.expand('prefix.html',
placeholders: &placeholders
missing_placeholder_prefix: '?'
)
assert rendered == '<p>?unknown</p>\n'
}
fn test_recursive_include_returns_internal_server_error() {
mut manager := new_test_manager(false)
placeholders := map[string]string{}
rendered := manager.expand('recursive.html', placeholders: &placeholders)
assert rendered == internal_server_error
assert manager.compiled_template_count() == 0
}
fn test_html_compression_is_deterministic() {
mut manager := new_test_manager(true)
placeholders := {
'title': 'Compact'
}
rendered := manager.expand('compress.html', placeholders: &placeholders)
assert rendered == '<div><span>Compact</span></div>'
}
fn test_empty_template_renders_empty() {
mut manager := new_test_manager(false)
placeholders := map[string]string{}
rendered := manager.expand('empty.html', placeholders: &placeholders)
assert rendered == ''
}
fn test_xml_template_uses_html_rendering_mode() {
mut manager := new_test_manager(false)
placeholders := {
'title': '<escaped>'
}
rendered := manager.expand('feed.xml', placeholders: &placeholders)
assert rendered == '<feed><title>&lt;escaped&gt;</title></feed>\n'
}
fn test_custom_template_extension_map() {
mut manager := initialize(
template_dir: dtm2_test_root()
template_extensions: {
'.page': TemplateType.html
'note': TemplateType.text
}
)
html_placeholders := {
'title_#includehtml': '<span>custom</span><script>blocked</script>'
}
text_placeholders := {
'body_#includehtml': '<span>custom</span>'
}
html := manager.expand('custom.page', placeholders: &html_placeholders)
text := manager.expand('custom.note', placeholders: &text_placeholders)
assert html.contains('<span>custom</span>&lt;script&gt;blocked&lt;/script&gt;')
assert text == 'Note: &lt;span&gt;custom&lt;/span&gt;\n'
}
fn test_custom_template_extension_map_can_override_builtin_mapping() {
mut manager := initialize(
template_dir: dtm2_test_root()
compress_html: true
template_extensions: {
'.html': TemplateType.text
}
)
placeholders := {
'title': 'Text'
}
rendered := manager.expand('override.html', placeholders: &placeholders)
assert rendered == '<div>\n Text\n</div>\n'
}
fn test_json_extension_config_file() {
mut manager := initialize(
template_dir: dtm2_test_root()
extension_config_file: os.join_path(dtm2_test_root(), 'extensions.json')
)
placeholders := {
'title': '<Config>'
}
html := manager.expand('custom.view', placeholders: &placeholders)
text := manager.expand('custom.mail', placeholders: &placeholders)
assert html == '<section>&lt;Config&gt;</section>'
assert text == 'Subject: &lt;Config&gt;\n'
}
fn test_default_json_extension_config_file_is_loaded_from_template_dir() {
mut manager := initialize(
template_dir: dtm2_test_root()
compress_html: false
)
placeholders := {
'title': '<Auto>'
}
html := manager.expand('auto.fragment', placeholders: &placeholders)
text := manager.expand('auto.message', placeholders: &placeholders)
assert html == '<aside>&lt;Auto&gt;</aside>\n'
assert text == 'Auto: &lt;Auto&gt;\n'
}
fn test_invalid_json_extension_entries_are_ignored() {
mut manager := initialize(
template_dir: dtm2_test_root()
extension_config_file: os.join_path(dtm2_test_root(), 'invalid_extensions.json')
compress_html: false
)
placeholders := {
'title': '<Safe>'
}
ignored := manager.expand('bad.path', placeholders: &placeholders)
loaded := manager.expand('auto.message', placeholders: &placeholders)
assert ignored == internal_server_error
assert loaded == 'Auto: &lt;Safe&gt;\n'
}
fn test_missing_template_returns_internal_server_error() {
mut manager := new_test_manager(false)
placeholders := map[string]string{}
rendered := manager.expand('missing.html', placeholders: &placeholders)
assert rendered == internal_server_error
assert manager.compiled_template_count() == 0
}
fn test_invalid_template_extension_returns_internal_server_error() {
mut manager := new_test_manager(false)
placeholders := map[string]string{}
rendered := manager.expand('invalid.css', placeholders: &placeholders)
assert rendered == internal_server_error
assert manager.compiled_template_count() == 0
}