Initial project import

tools/v/vlib/x/templating/dtm/README.md

@@ -0,0 +1,292 @@
+# dtm - Dynamic Template Manager
+
+A simple template manager integrated into the V project, designed to combine the power of V
+templates with Veb, without the need to recompile the application with every change.
+
+> **Status:** `x.templating.dtm` is the legacy version of Dynamic Template
+> Manager. New projects should use `x.templating.dtm2`, which provides the
+> modern runtime renderer and parsed-template cache.
+>
+> Existing `dtm` users can keep their current API while migrating progressively:
+> the v1 facade now delegates rendering to the DTM2 engine internally, preserving
+> source compatibility while removing the old rendered-cache server from the
+> main rendering path.
+
+## Quick Start
+
+Using the dynamic template manager (named `dtm` in this readme) is relatively straightforward.
+You just need to initialize an instance. Then, call the `expand` function in the code that
+manages your template.
+
+Before starting, you can specify a folder, but it is not mandatory to store the generated cache. If
+nothing is specified or if there is a problem with the targeted folder (for example, a permission
+issue), the DTM will attempt to create a cache folder in the temporary file area of your OS.
+Finally, if all this does not prove successful, then the DTM will disable the cache system and
+notify the user if the cache system was previously required.
+
+However, at the root directory of your project, you need to create a `templates/` folder. The
+absence of these directories will prevent you from using the dtm. You must add your template files
+into the folder you previously created. ***Be aware that if the templates are not placed in the
+correct directory, the DTM will return an error message indicating that it cannot find the
+template!***
+
+The DTM currently only works with 2 types of files:
+
+- html
+- raw text
+
+Below you will find 2 usage contexts:
+
+### 1. Minimal static generator example :
+
+```v
+import x.templating.dtm
+
+fn main() {
+	mut dtmi := dtm.initialize()
+
+	// No need to add this 'defer' if you have chosen to disable the cache system in the options.
+	defer {
+		dtmi.stop_cache_handler()
+	}
+	mut tmp_var := map[string]dtm.DtmMultiTypeMap{}
+	tmp_var['title'] = 'V dtm best title'
+	tmp_var['non_string_type'] = 7
+	tmp_var['html_section_#includehtml'] = '<span>will <br> be <br> escaped <br> in <br> text mode</span>'
+
+	render := dtmi.expand('test.txt', placeholders: &tmp_var)
+
+	println(render)
+}
+```
+
+#### and its template text format:
+
+```
+Title of text: @title
+Value in the text: @non_string_type
+HTML tags are always escaped in text file: @html_section
+```
+
+### 2. Minimal Veb example:
+
+```v
+import veb
+import x.templating.dtm
+import os
+
+pub struct App {
+pub mut:
+	dtmi &dtm.DynamicTemplateManager = unsafe { nil }
+}
+
+pub struct Context {
+	veb.Context
+}
+
+fn main() {
+	cache_folder_path := os.join_path(os.dir(os.executable()), 'vcache_dtm')
+	mut app := &App{
+		dtmi: dtm.initialize(def_cache_path: cache_folder_path)
+	}
+	// No need to add this 'defer' if you have chosen to disable the cache system in the options.
+	defer {
+		app.dtmi.stop_cache_handler()
+	}
+
+	// Here is an example of init configuration:
+
+	dtm.initialize(
+		def_cache_path:       cache_folder_path
+		compress_html:        false
+		active_cache_server:  false
+		max_size_data_in_mem: 100
+	)
+
+	veb.run[App, Context](mut app, 18081)
+}
+
+@['/']
+pub fn (mut app App) index(mut ctx Context) veb.Result {
+	mut tmpl_var := map[string]dtm.DtmMultiTypeMap{}
+	tmpl_var['title'] = 'The true title'
+	html_content := app.dtmi.expand('index.html', placeholders: &tmpl_var)
+	return ctx.html(html_content)
+}
+```
+
+#### and its template html format:
+
+```html
+<!doctype html>
+<html>
+	<head>
+		<title>@title</title>
+	</head>
+	<body>
+		<div id="container">
+			<h1>@title</h1>
+		</div>
+	</body>
+</html>
+```
+
+You have a ready-to-view demonstration available
+[here](https://github.com/vlang/v/tree/master/vlib/veb/tests/dynamic_template_manager_test_server).
+
+## Available Options
+
+There are two types of option possibilities:
+
+- Specific to the initialization
+- Defined for each call of expand function
+
+### Specific to the initialization
+
+Three parameters are available:
+
+- `def_cache_path` (**string** value): User can define the path of cache folder.
+- `max_size_data_in_mem` (**int** value): Maximum size of data allowed in memory for each cached.
+  template. The value must be specified in kilobytes (default is: 500KB; limit max is: 500KB).
+- `compress_html` (**bool** value): Light **minifier** of the HTML output, to remove all
+unnecessary spacing (default is `true`, parameter taken into account only for HTML files).
+- `active_cache_server` (**bool** value): Activate or not the template cache system (default
+  is true, ***_highly recommended to keep it enabled for optimal performance_***).
+
+Regarding the `compress_html` option, it is recommended for performance reasons to disable it
+when working directly with raw template generation (i.e., with the cache system disabled).
+
+Use it like this:
+
+```v ignore
+initialize(
+    def_cache_path: 'your/directorie/cache/path'
+    max_size_data_in_mem: 500
+    compress_html: true
+    active_cache_server: true
+)
+```
+
+### Defined for each call of expand function
+
+- `placeholders` (**&map[string]DtmMultiTypeMap** value): Used to map placeholders within the
+  template to their corresponding content, facilitating dynamic content insertion, by specifying
+  values in the placeholders map. Templates can dynamically display content.
+
+- `cache_delay_expiration` (**i64** value): Specifies the cache expiration time for the concerned
+  page in seconds (default value is `86400` seconds or one day). You can add any value you
+  want in seconds as long as it remains within the indicated range (see below).
+
+Possibility to use already defined cache delay constants like:
+
+- `cache_delay_expiration_at_min`: five minutes.
+- `cache_delay_expiration_at_max`: one year.
+- `cache_delay_expiration_by_default`: one day.
+
+For specific cases, you can cancel the generation and use of cache file, even if the cache system
+is active:
+
+- `cache_delay_expiration: -1`.
+
+Or set a cache that will never expire:
+
+- `cache_delay_expiration: 0`.
+
+Example:
+
+```v ignore
+expand('path/of/template.html',
+		placeholders: &the_map_var
+		cache_delay_expiration: -1
+)
+```
+
+## The PlaceHolders System
+
+### On The User's Side Code:
+
+The placeholder system allows for the insertion of dynamic content into your template. As of the
+current state of the module, it accepts the following types like:
+
+```
+- string
+- i8, i16, int, i64
+- u8, u16, u32, u64
+- f32, f64
+```
+
+Example:
+
+```v ignore
+mut plhs := map[string]dtm.DtmMultiTypeMap{}
+plhs['placeholder_name_1'] = "title content"
+plhs['placeholder_name_2'] = 123456
+plhs['placeholder_name_3_#includehtml'] = "<p>allow to include</p><span>certain HTML tags</span>"
+
+expand('path/of/template.html',
+		placeholders: &plhs
+)
+```
+
+Pay attention to this particular tag: `_#includehtml`, it enables you to include HTML in the
+dynamic content. Without this tag, all characters will be escaped for obvious security reasons. By
+using this tag, only certain HTML tags are allowed. Here is the list:
+
+```html
+'<div>', '</div>', '<h1>', '</h1>', '<h2>', '</h2>', '<h3>', '</h3>', '<h4>',
+'</h4>', '<h5>', '</h5>', '<h6>', '</h6>', '<p>', '</p>', '<br>', '<hr>', '<span>', '</span>',
+'<ul>', '</ul>', '<ol>', '</ol>', '<li>', '</li>', '<dl>', '</dl>', '<dt>', '</dt>', '<dd>',
+'</dd>', '<menu>', '</menu>', '<table>', '</table>', '<caption>', '</caption>', '<th>', '</th>',
+'<tr>', '</tr>', '<td>', '</td>', '<thread>', '</thread>', '<tbody>', '</tbody>', '<tfoot>',
+'</tfoot>', '<col>', '</col>', '<colgroup>', '</colgroup>', '<header>', '</header>', '<footer>',
+'</footer>', '<main>', '</main>', '<section>', '</section>', '<article>', '</article>', '<aside>',
+'</aside>', '<details>', '</details>', '<dialog>', '</dialog>', '<data>', '</data>', '<summary>',
+'</summary>'
+```
+
+#### Note that with a raw text template, all HTML tag inclusions are escaped.
+
+### On The Template Side:
+
+An example of a template, corresponding to the previous subsection:
+
+```html
+<!doctype html>
+<html>
+	<head>
+		<title>@placeholder_name_1</title>
+	</head>
+	<body>
+		<div id="container">
+			<h1>@placeholder_name_1</h1>
+			<p>@placeholder_name_2</p>
+			@placeholder_name_3
+		</div>
+	</body>
+</html>
+```
+
+You will note that the `_#includehtml` directive is not found in the template with
+`@placeholder_name_3`, and this is entirely normal. Directives are specially handled
+by the DTM, and including them in the name of your placeholders within the template
+will result in the placeholder not being found because it does not match the key name
+defined in the map containing the dynamic content.
+
+Like the traditional template system in V, inclusions or placeholders start with the `@`
+character. The traditional inclusion system is still perfectly usable, such as:
+
+```
+- @include 'my/html/path.html'
+- @css 'my/css/path.css'
+- @js 'my/js/path.js'
+```
+
+## In The Future
+
+As you've understood, the DTM is still under development and optimization. There are
+functionalities to be added, such as data compression, managing loops or conditions within the
+template itself. Able to be used in contexts other than HTML and raw text.
+
+This will come in time.
+
+Feel free to report any issues or contribute to the project!

tools/v/vlib/x/templating/dtm/dynamic_template_manager_behavior_test.v

@@ -0,0 +1,333 @@
+// vtest retry: 3
+module dtm
+
+import os
+import time
+
+const behavior_temp_dtm_dir = 'dynamic_template_manager_behavior_test'
+const behavior_temp_cache_dir = 'vcache_dtm'
+const behavior_temp_templates_dir = 'templates'
+const behavior_vtmp_dir = os.vtmp_dir()
+
+fn behavior_test_root_dir() string {
+	return os.join_path(behavior_vtmp_dir, '${behavior_temp_dtm_dir}_${os.getpid()}')
+}
+
+fn testsuite_begin() {
+	temp_folder := behavior_test_root_dir()
+	os.rmdir_all(temp_folder) or {}
+	os.mkdir_all(os.join_path(temp_folder, behavior_temp_cache_dir))!
+	os.mkdir_all(os.join_path(temp_folder, behavior_temp_templates_dir))!
+
+	templates_path := os.join_path(temp_folder, behavior_temp_templates_dir)
+	os.write_file(os.join_path(templates_path, 'page.html'), [
+		'<!doctype html>',
+		'<title>@title</title>',
+		'<p>@body</p>',
+		'<p>@count</p>',
+	].join('\n'))!
+	os.write_file(os.join_path(templates_path, 'page.txt'), [
+		'Title: @title',
+		'Body: @body',
+	].join('\n'))!
+	os.write_file(os.join_path(templates_path, 'layout.html'), [
+		'<header>Start</header>',
+		"@include 'partial'",
+		'<footer>End</footer>',
+	].join('\n'))!
+	os.write_file(os.join_path(templates_path, 'partial.html'), '<section>@title</section>')!
+	os.write_file(os.join_path(templates_path, 'cache.html'), '<main>@title</main>')!
+	os.write_file(os.join_path(templates_path, 'cache_file_update.html'), '<main>@title</main>')!
+	os.write_file(os.join_path(templates_path, 'no_store.html'), '<main>@title</main>')!
+	os.write_file(os.join_path(templates_path, 'forever.html'), '<main>@title</main>')!
+	os.write_file(os.join_path(templates_path, 'disk_cache.html'), '<main>@title</main>')!
+	os.write_file(os.join_path(templates_path, 'include_html.html'), '<article>@content</article>')!
+	os.write_file(os.join_path(templates_path, 'include_text.txt'), 'Text: @content')!
+	os.write_file(os.join_path(templates_path, 'invalid.css'), 'body { color: red; }')!
+}
+
+fn testsuite_end() {
+	os.rmdir_all(behavior_test_root_dir()) or {}
+}
+
+fn new_behavior_dtm(active_cache_server bool, compress_html bool, max_size_data_in_mem int) &DynamicTemplateManager {
+	temp_folder := behavior_test_root_dir()
+	return initialize(
+		active_cache_server:  active_cache_server
+		compress_html:        compress_html
+		max_size_data_in_mem: max_size_data_in_mem
+		test_cache_dir:       os.join_path(temp_folder, behavior_temp_cache_dir)
+		test_template_dir:    os.join_path(temp_folder, behavior_temp_templates_dir)
+	)
+}
+
+fn behavior_template_path(file_name string) string {
+	return os.join_path(behavior_test_root_dir(), behavior_temp_templates_dir, file_name)
+}
+
+fn rendered_cache_count(dtmi &DynamicTemplateManager) int {
+	rlock dtmi.template_caches {
+		return dtmi.template_caches.len
+	}
+}
+
+fn test_expand_escapes_html_placeholders_by_default() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'title': DtmMultiTypeMap('Hello <V>')
+		'body':  DtmMultiTypeMap('<script>alert(1)</script>')
+		'count': DtmMultiTypeMap(7)
+	}
+
+	rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<title>Hello &lt;V&gt;</title>')
+	assert rendered.contains('<p>&lt;script&gt;alert(1)&lt;/script&gt;</p>')
+	assert rendered.contains('<p>7</p>')
+}
+
+fn test_expand_does_not_double_escape_placeholder_values_containing_dollar() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'title': DtmMultiTypeMap('Cost $5 <V>')
+		'body':  DtmMultiTypeMap('Body')
+		'count': DtmMultiTypeMap(1)
+	}
+
+	rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<title>Cost $5 &lt;V&gt;</title>')
+}
+
+fn test_expand_escapes_html_entities_and_quotes() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'title': DtmMultiTypeMap('A & B "quoted" \'single\'')
+		'body':  DtmMultiTypeMap('Body')
+		'count': DtmMultiTypeMap(1)
+	}
+
+	rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<title>A &amp; B &#34;quoted&#34; &#39;single&#39;</title>')
+}
+
+fn test_expand_escapes_text_templates() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'title': DtmMultiTypeMap('Plain <Title>')
+		'body':  DtmMultiTypeMap('<strong>raw text</strong>')
+	}
+
+	rendered := dtmi.expand(behavior_template_path('page.txt'), placeholders: &placeholders)
+
+	assert rendered.contains('Title: Plain &lt;Title&gt;')
+	assert rendered.contains('Body: &lt;strong&gt;raw text&lt;/strong&gt;')
+}
+
+fn test_expand_compresses_html_without_regex_failure() {
+	mut dtmi := new_behavior_dtm(false, true, max_size_data_in_memory)
+	placeholders := {
+		'title': DtmMultiTypeMap('Compressed')
+		'body':  DtmMultiTypeMap('Body')
+		'count': DtmMultiTypeMap(1)
+	}
+
+	rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
+
+	assert rendered != internat_server_error
+	assert !rendered.contains('\n')
+	assert rendered.contains('<!doctype html><title>Compressed</title><p>Body</p><p>1</p>')
+}
+
+fn test_expand_with_empty_placeholders_keeps_template_renderable() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := map[string]DtmMultiTypeMap{}
+
+	rendered := dtmi.expand(behavior_template_path('page.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<title>$title</title>')
+	assert rendered.contains('<p>$body</p>')
+	assert rendered.contains('<p>$count</p>')
+}
+
+fn test_expand_resolves_relative_includes() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'title': DtmMultiTypeMap('Included <Title>')
+	}
+
+	rendered := dtmi.expand(behavior_template_path('layout.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<header>Start</header>')
+	assert rendered.contains('<section>Included &lt;Title&gt;</section>')
+	assert rendered.contains('<footer>End</footer>')
+}
+
+fn test_includehtml_preserves_allowed_tags_and_escapes_unlisted_tags() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'content_#includehtml': DtmMultiTypeMap('<span>allowed</span><script>blocked()</script>')
+	}
+
+	rendered :=
+		dtmi.expand(behavior_template_path('include_html.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<article><span>allowed</span>&lt;script&gt;blocked()&lt;/script&gt;</article>')
+}
+
+fn test_includehtml_is_still_escaped_in_text_templates() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := {
+		'content_#includehtml': DtmMultiTypeMap('<span>text</span>')
+	}
+
+	rendered := dtmi.expand(behavior_template_path('include_text.txt'), placeholders: &placeholders)
+
+	assert rendered.contains('Text: &lt;span&gt;text&lt;/span&gt;')
+}
+
+fn test_expand_returns_internal_error_for_missing_template() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := map[string]DtmMultiTypeMap{}
+
+	rendered := dtmi.expand(behavior_template_path('missing.html'), placeholders: &placeholders)
+
+	assert rendered == internat_server_error
+}
+
+fn test_expand_returns_internal_error_for_invalid_template_extension() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	placeholders := map[string]DtmMultiTypeMap{}
+
+	rendered := dtmi.expand(behavior_template_path('invalid.css'), placeholders: &placeholders)
+
+	assert rendered == internat_server_error
+}
+
+fn test_expand_without_cache_renders_fresh_placeholder_content() {
+	mut dtmi := new_behavior_dtm(false, false, max_size_data_in_memory)
+	first_placeholders := {
+		'title': DtmMultiTypeMap('first')
+	}
+	second_placeholders := {
+		'title': DtmMultiTypeMap('second')
+	}
+
+	first := dtmi.expand(behavior_template_path('cache.html'), placeholders: &first_placeholders)
+	second := dtmi.expand(behavior_template_path('cache.html'), placeholders: &second_placeholders)
+
+	assert first.contains('<main>first</main>')
+	assert second.contains('<main>second</main>')
+}
+
+fn test_expand_with_cache_updates_when_placeholder_content_changes() {
+	mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
+	defer {
+		dtmi.disable_cache()
+	}
+	first_placeholders := {
+		'title': DtmMultiTypeMap('cached first')
+	}
+	second_placeholders := {
+		'title': DtmMultiTypeMap('cached second')
+	}
+
+	first := dtmi.expand(behavior_template_path('cache.html'), placeholders: &first_placeholders)
+	second := dtmi.expand(behavior_template_path('cache.html'), placeholders: &first_placeholders)
+	third := dtmi.expand(behavior_template_path('cache.html'), placeholders: &second_placeholders)
+
+	assert first.contains('<main>cached first</main>')
+	assert second == first
+	assert third.contains('<main>cached second</main>')
+}
+
+fn test_expand_with_legacy_cache_enabled_uses_new_parsed_template_engine() {
+	mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
+	defer {
+		dtmi.disable_cache()
+	}
+	placeholders := {
+		'title': DtmMultiTypeMap('stable cached value')
+	}
+
+	first := dtmi.expand(behavior_template_path('cache.html'), placeholders: &placeholders)
+	second := dtmi.expand(behavior_template_path('cache.html'), placeholders: &placeholders)
+
+	assert first == second
+	assert rendered_cache_count(dtmi) == 0
+	assert dtmi.render_engine.compiled_template_count() == 1
+}
+
+fn test_expand_with_cache_updates_when_template_file_changes() {
+	mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
+	defer {
+		dtmi.disable_cache()
+	}
+	template_path := behavior_template_path('cache_file_update.html')
+	placeholders := {
+		'title': DtmMultiTypeMap('file driven')
+	}
+
+	first := dtmi.expand(template_path, placeholders: &placeholders)
+	time.sleep(1100 * time.millisecond)
+	os.write_file(template_path, '<section>@title</section>')!
+	second := dtmi.expand(template_path, placeholders: &placeholders)
+
+	assert first.contains('<main>file driven</main>')
+	assert second.contains('<section>file driven</section>')
+}
+
+fn test_cache_delay_minus_one_skips_rendered_cache_storage() {
+	mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
+	defer {
+		dtmi.disable_cache()
+	}
+	placeholders := {
+		'title': DtmMultiTypeMap('no store')
+	}
+
+	rendered := dtmi.expand(behavior_template_path('no_store.html'),
+		placeholders:           &placeholders
+		cache_delay_expiration: -1
+	)
+
+	assert rendered.contains('<main>no store</main>')
+	assert rendered_cache_count(dtmi) == 0
+}
+
+fn test_cache_delay_zero_is_accepted_without_rendered_cache_storage() {
+	mut dtmi := new_behavior_dtm(true, false, max_size_data_in_memory)
+	defer {
+		dtmi.disable_cache()
+	}
+	placeholders := {
+		'title': DtmMultiTypeMap('forever')
+	}
+
+	rendered := dtmi.expand(behavior_template_path('forever.html'),
+		placeholders:           &placeholders
+		cache_delay_expiration: 0
+	)
+
+	assert rendered.contains('<main>forever</main>')
+	assert rendered_cache_count(dtmi) == 0
+}
+
+fn test_memory_limit_zero_does_not_create_rendered_disk_cache() {
+	mut dtmi := new_behavior_dtm(true, false, 0)
+	defer {
+		dtmi.disable_cache()
+	}
+	placeholders := {
+		'title': DtmMultiTypeMap('disk mode')
+	}
+
+	rendered := dtmi.expand(behavior_template_path('disk_cache.html'), placeholders: &placeholders)
+
+	assert rendered.contains('<main>disk mode</main>')
+	assert rendered_cache_count(dtmi) == 0
+	for file_name in os.ls(dtmi.template_cache_folder)! {
+		assert !file_name.ends_with('.cache')
+	}
+}

tools/v/vlib/x/templating/dtm/dynamic_template_manager_cache_system_test.v

@@ -0,0 +1,176 @@
+// vtest retry: 3
+module dtm
+
+import os
+import time
+
+const temp_dtm_dir = 'dynamic_template_manager_cache_system_test'
+const temp_cache_dir = 'vcache_dtm'
+const temp_templates_dir = 'templates'
+const temp_html_fp = 'temp.html'
+const temp_html_n = 'temp'
+const vtmp_dir = os.vtmp_dir()
+
+fn testsuite_begin() {
+	temp_folder := os.join_path(vtmp_dir, temp_dtm_dir)
+	os.mkdir_all(temp_folder)!
+
+	vcache_path := os.join_path(temp_folder, temp_cache_dir)
+	templates_path := os.join_path(temp_folder, temp_templates_dir)
+
+	os.mkdir_all(vcache_path)!
+	os.mkdir_all(templates_path)!
+
+	temp_html_file := os.join_path(templates_path, temp_html_fp)
+
+	html_content := '
+    <!DOCTYPE html>
+    <html>
+      <head>
+        <title>TEST</title>
+      </head>
+      <body>
+        <div>
+          <H1>TEST</H1>
+        </div>
+      </body>
+    </html>'
+
+	os.write_file(temp_html_file, html_content)!
+}
+
+fn test_check_and_clear_cache_files() {
+	dtmi := init_dtm(false, 0)
+	check_and_clear_cache_files(dtmi.template_cache_folder)!
+
+	count_cache_files := os.ls(dtmi.template_cache_folder)!
+	assert count_cache_files.len == 0
+}
+
+fn test_create_template_cache_and_display() {
+	mut dtmi := init_dtm(true, max_size_data_in_memory)
+	defer {
+		dtmi.stop_cache_handler()
+	}
+	html := dtmi.create_cache()
+	assert html.len > 10
+}
+
+fn test_get_cache() {
+	mut dtmi := init_dtm(true, max_size_data_in_memory)
+	dtmi.create_cache()
+	defer {
+		dtmi.stop_cache_handler()
+	}
+	dtm_placeholders := map[string]DtmMultiTypeMap{}
+	temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
+	html_mem := dtmi.get_cache(temp_html_n, temp_html_file, &dtm_placeholders)
+	assert html_mem.len > 10
+}
+
+fn test_chandler_clear_specific_cache() {
+	mut dtmi := init_dtm(true, 0)
+	defer {
+		dtmi.stop_cache_handler()
+	}
+	dtmi.create_cache()
+	lock dtmi.template_caches {
+		cache_file := os.join_path(dtmi.template_cache_folder,
+			'${dtmi.template_caches[0].name}_${dtmi.template_caches[0].checksum}.cache')
+		index, is_success := dtmi.chandler_clear_specific_cache(dtmi.template_caches[0].id)
+		assert is_success == true
+		assert index == 0
+		cache_exist := os.exists(cache_file)
+		assert cache_exist == false
+	}
+}
+
+fn test_cache_timestamp_is_initialized() {
+	mut dtmi := init_dtm(true, max_size_data_in_memory)
+	defer {
+		dtmi.stop_cache_handler()
+	}
+	date_to_str := dtmi.c_time.str()
+	assert date_to_str.len > 10
+}
+
+fn test_process_rendered_cache_delete_request() {
+	mut dtmi := init_dtm(true, max_size_data_in_memory)
+	defer {
+		dtmi.stop_cache_handler()
+	}
+	dtmi.create_cache()
+	path_f := os.join_path(dtmi.template_folder, temp_html_fp)
+	lock dtmi.template_caches {
+		assert dtmi.template_caches[0].id == 1
+		assert dtmi.template_caches[0].name == temp_html_n
+		assert dtmi.template_caches[0].path == path_f
+	}
+	dtmi.id_to_handlered = 1
+	dtmi.ch_cache_handler <- TemplateCache{
+		id:            1
+		cache_request: .delete
+	}
+	dtmi.sync_cache()
+	rlock dtmi.template_caches {
+		assert dtmi.template_caches.len == 0
+	}
+}
+
+fn testsuite_end() {
+	temp_folder := os.join_path(vtmp_dir, temp_dtm_dir)
+	os.rmdir_all(temp_folder) or {}
+}
+
+// Utilities function :
+
+fn init_dtm(b bool, m int) &DynamicTemplateManager {
+	temp_folder := os.join_path(vtmp_dir, temp_dtm_dir)
+	vcache_path := os.join_path(temp_folder, temp_cache_dir)
+	templates_path := os.join_path(temp_folder, temp_templates_dir)
+
+	init_params := DynamicTemplateManagerInitialisationParams{
+		active_cache_server:  b
+		max_size_data_in_mem: m
+		test_cache_dir:       vcache_path
+		test_template_dir:    templates_path
+	}
+
+	dtm := initialize(init_params)
+
+	return dtm
+}
+
+fn (mut tm DynamicTemplateManager) create_cache() string {
+	temp_html_file := os.join_path(tm.template_folder, temp_html_fp)
+	html_last_mod := os.file_last_mod_unix(temp_html_file)
+	c_time := get_current_unix_micro_timestamp()
+	cache_delay_exp := i64(500) * i64(1000000)
+	placeholder := map[string]DtmMultiTypeMap{}
+	content_checksum := ''
+	html := tm.create_template_cache_and_display(.new, html_last_mod, c_time, temp_html_file,
+		temp_html_n, cache_delay_exp, &placeholder, content_checksum, TemplateType.html)
+	tm.sync_cache()
+
+	return html
+}
+
+fn (mut tm DynamicTemplateManager) sync_cache() {
+	mut count := 0
+	for {
+		select {
+			_ := <-tm.is_ready {
+				break
+			}
+			else {
+				time.sleep(50 * time.millisecond)
+				// Wait a total of 2 seconds to check if the data cache is ready
+				if count >= 40 {
+					tm.abort_test = true
+					break
+				}
+				count++
+			}
+		}
+	}
+}

tools/v/vlib/x/templating/dtm/dynamic_template_manager_dtm2_bridge.v

@@ -0,0 +1,65 @@
+module dtm
+
+import x.templating.dtm2
+
+// new_dtm2_render_engine is the compatibility boundary between the historical
+// DTM API and the DTM2 runtime template engine. DTM2 remains unaware
+// of legacy cache concepts; it only parses templates and renders them.
+fn new_dtm2_render_engine(template_dir string, compress_html bool) &dtm2.Manager {
+	return dtm2.initialize(
+		template_dir:              template_dir
+		compress_html:             compress_html
+		reload_modified_templates: true
+	)
+}
+
+fn (mut tm DynamicTemplateManager) expand_with_dtm2_render_engine(tmpl_path string, tmpl_var TemplateCacheParams) string {
+	validate_legacy_placeholder_sizes(tmpl_path, tmpl_var.placeholders) or {
+		eprintln(err.msg())
+		return internat_server_error
+	}
+	if isnil(tm.render_engine) {
+		tm.render_engine = new_dtm2_render_engine(tm.template_folder, tm.compress_html)
+	}
+	placeholders := convert_legacy_placeholders(tmpl_var.placeholders)
+	return tm.render_engine.expand(tmpl_path,
+		placeholders:               &placeholders
+		missing_placeholder_prefix: '$'
+	)
+}
+
+fn validate_legacy_placeholder_sizes(tmpl_path string, placeholders &map[string]DtmMultiTypeMap) ! {
+	for key, value in placeholders {
+		if key.len > max_placeholders_key_size {
+			return error('${message_signature_error} Length of placeholder key "${key}" exceeds the maximum allowed size for template content in file: ${tmpl_path}. Max allowed size: ${max_placeholders_key_size} characters.')
+		}
+		casted_value := legacy_placeholder_value_to_string(*value)
+		if casted_value.len > max_placeholders_value_size {
+			return error('${message_signature_error} Length of placeholder value for key "${key}" exceeds the maximum allowed size for template content in file: ${tmpl_path}. Max allowed size: ${max_placeholders_value_size} characters.')
+		}
+	}
+}
+
+fn convert_legacy_placeholders(placeholders &map[string]DtmMultiTypeMap) map[string]string {
+	mut converted := map[string]string{}
+	for key, value in placeholders {
+		converted[key.clone()] = legacy_placeholder_value_to_string(*value).clone()
+	}
+	return converted
+}
+
+fn legacy_placeholder_value_to_string(value DtmMultiTypeMap) string {
+	return match value {
+		f32 { value.str() }
+		f64 { value.str() }
+		i16 { value.str() }
+		i64 { value.str() }
+		i8 { value.str() }
+		int { value.str() }
+		string { value.clone() }
+		u16 { value.str() }
+		u32 { value.str() }
+		u64 { value.str() }
+		u8 { value.str() }
+	}
+}

tools/v/vlib/x/templating/dtm/dynamic_template_manager_test.v

@@ -0,0 +1,342 @@
+module dtm
+
+import os
+
+const temp_dtm_dir = 'dynamic_template_manager_test'
+const temp_templates_dir = 'templates'
+const temp_html_fp = 'temp.html'
+const temp_html_n = 'temp'
+const vtmp_dir = os.vtmp_dir()
+
+fn dtm_test_root_dir() string {
+	return os.join_path(vtmp_dir, '${temp_dtm_dir}_${os.getpid()}')
+}
+
+fn testsuite_begin() {
+	temp_folder := dtm_test_root_dir()
+	os.rmdir_all(temp_folder) or {}
+	os.mkdir_all(temp_folder)!
+
+	templates_path := os.join_path(temp_folder, temp_templates_dir)
+
+	os.mkdir_all(templates_path)!
+
+	temp_html_file := os.join_path(templates_path, temp_html_fp)
+
+	html_content := '
+    <!DOCTYPE html>
+    <html>
+      <head>
+        <title>TEST</title>
+      </head>
+      <body>
+        <div>
+          <H1>TEST</H1>
+        </div>
+      </body>
+    </html>'
+
+	os.write_file(temp_html_file, html_content)!
+}
+
+fn test_initialize_dtm() {
+	dtmi := init_dtm(false, 0)
+	assert dtmi.dtm_init_is_ok == true
+}
+
+fn test_create_template_cache_and_display() {
+	mut dtmi := init_dtm(false, 0)
+	temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
+	html_last_mod := os.file_last_mod_unix(temp_html_file)
+	c_time := get_current_unix_micro_timestamp()
+	cache_delay_exp := i64(500) * i64(1000000)
+	placeholder := map[string]DtmMultiTypeMap{}
+	content_checksum := ''
+	html := dtmi.create_template_cache_and_display(.new, html_last_mod, c_time, temp_html_file,
+		temp_html_n, cache_delay_exp, &placeholder, content_checksum, TemplateType.html)
+
+	assert html.len > 10
+}
+
+fn test_return_cache_info_isexistent() {
+	mut dtmi := init_dtm(false, 0)
+	path_template := os.join_path(dtmi.template_folder, temp_html_fp)
+	lock dtmi.template_caches {
+		dtmi.template_caches << TemplateCache{
+			id:   1
+			path: path_template
+		}
+	}
+	lock dtmi.nbr_of_remaining_template_request {
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id: 1
+		}
+	}
+	cache_exists, _, _, _, _, _, _ := dtmi.return_cache_info_isexistent(path_template)
+	assert cache_exists == true
+	lock dtmi.template_caches {
+		dtmi.template_caches[0].id_redirection = 2
+		dtmi.template_caches << TemplateCache{
+			id:             2
+			path:           path_template
+			id_redirection: 3
+		}
+		dtmi.template_caches << TemplateCache{
+			id:             3
+			path:           path_template
+			id_redirection: 4
+		}
+		dtmi.template_caches << TemplateCache{
+			id:             4
+			path:           path_template
+			id_redirection: 5
+		}
+		dtmi.template_caches << TemplateCache{
+			id:   5
+			path: path_template
+		}
+	}
+	lock dtmi.nbr_of_remaining_template_request {
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id: 2
+		}
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id: 3
+		}
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id: 4
+		}
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id: 5
+		}
+	}
+	_, id, _, _, _, _, _ := dtmi.return_cache_info_isexistent(path_template)
+	assert id == 5
+}
+
+fn test_remaining_template_request() {
+	mut dtmi := init_dtm(false, 0)
+
+	lock dtmi.nbr_of_remaining_template_request {
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id: 1
+		}
+	}
+	dtmi.remaining_template_request(true, 1)
+	rlock dtmi.nbr_of_remaining_template_request {
+		assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 1
+	}
+	dtmi.remaining_template_request(true, 1)
+	rlock dtmi.nbr_of_remaining_template_request {
+		assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 2
+	}
+	dtmi.remaining_template_request(false, 1)
+	rlock dtmi.nbr_of_remaining_template_request {
+		assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 1
+	}
+	dtmi.remaining_template_request(false, 1)
+	rlock dtmi.nbr_of_remaining_template_request {
+		assert dtmi.nbr_of_remaining_template_request[0].nbr_of_remaining_request == 0
+	}
+}
+
+fn test_check_tmpl_and_placeholders_size() {
+	mut dtmi := init_dtm(false, 0)
+	temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
+	placeholders := map[string]DtmMultiTypeMap{}
+
+	path, filename, content_checksum, tmpl_type := dtmi.check_tmpl_and_placeholders_size(temp_html_file,
+		&placeholders)!
+
+	is_html := if tmpl_type == TemplateType.html {
+		true
+	} else {
+		false
+	}
+	assert path.len > 10
+	assert filename.len > 3
+	assert is_html == true
+	//	assert content_checksum.len > 3
+}
+
+fn test_chandler_prevent_cache_duplicate_request() {
+	mut dtmi := init_dtm(false, 0)
+	temp_html_file := os.join_path(dtmi.template_folder, temp_html_fp)
+
+	lock dtmi.template_caches {
+		dtmi.template_caches << TemplateCache{
+			id:            1
+			path:          temp_html_file
+			cache_request: .new
+		}
+		dtmi.template_caches << TemplateCache{
+			id:                2
+			path:              temp_html_file
+			cache_request:     .update
+			last_template_mod: i64(1)
+		}
+		dtmi.template_caches << TemplateCache{
+			id:                3
+			path:              temp_html_file
+			cache_request:     .exp_update
+			last_template_mod: i64(1)
+			generate_at:       i64(100)
+		}
+		dtmi.template_caches << TemplateCache{
+			id:            4
+			cache_request: .delete
+		}
+	}
+	new_cache := TemplateCache{
+		id:            5
+		path:          temp_html_file
+		cache_request: .new
+	}
+	update_cache := TemplateCache{
+		id:                6
+		path:              temp_html_file
+		cache_request:     .update
+		last_template_mod: i64(1)
+	}
+	exp_update_cache := TemplateCache{
+		id:                     7
+		path:                   temp_html_file
+		cache_request:          .exp_update
+		last_template_mod:      i64(1)
+		generate_at:            i64(10)
+		cache_delay_expiration: i64(10)
+	}
+	delete_cache := TemplateCache{
+		id:            4
+		cache_request: .delete
+	}
+	mut is_duplicate := dtmi.chandler_prevent_cache_duplicate_request(&new_cache)
+	assert is_duplicate == true
+	is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&update_cache)
+	assert is_duplicate == true
+	is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&exp_update_cache)
+	assert is_duplicate == true
+	is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&delete_cache)
+	assert is_duplicate == false
+
+	lock dtmi.template_caches {
+		dtmi.template_caches.delete(3)
+	}
+
+	is_duplicate = dtmi.chandler_prevent_cache_duplicate_request(&delete_cache)
+	assert is_duplicate == true
+}
+
+fn test_chandler_remaining_cache_template_used() {
+	mut dtmi := init_dtm(false, 0)
+	lock dtmi.nbr_of_remaining_template_request {
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id:                       1
+			nbr_of_remaining_request: 0
+		}
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id:                       2
+			nbr_of_remaining_request: 1
+			need_to_delete:           true
+		}
+		dtmi.nbr_of_remaining_template_request << RemainingTemplateRequest{
+			id:                       3
+			nbr_of_remaining_request: 0
+			need_to_delete:           true
+		}
+	}
+	mut can_delete := dtmi.chandler_remaining_cache_template_used(CacheRequest.update, 3, 3)
+	assert can_delete == true
+	can_delete = dtmi.chandler_remaining_cache_template_used(CacheRequest.update, 2, 2)
+	assert can_delete == false
+	can_delete = dtmi.chandler_remaining_cache_template_used(CacheRequest.delete, 1, 0)
+	assert can_delete == true
+	can_delete = dtmi.chandler_remaining_cache_template_used(CacheRequest.new, 4, 0)
+	assert can_delete == true
+}
+
+fn test_parse_tmpl_file() {
+	mut dtmi := init_dtm(false, 0)
+	temp_folder := dtm_test_root_dir()
+	templates_path := os.join_path(temp_folder, temp_templates_dir)
+	temp_html_file := os.join_path(templates_path, temp_html_fp)
+
+	mut placeholders := map[string]DtmMultiTypeMap{}
+
+	is_compressed := true
+	html := dtmi.parse_tmpl_file(temp_html_file, temp_html_n, &placeholders, is_compressed,
+		TemplateType.html)
+
+	assert html.len > 0
+}
+
+fn test_check_if_cache_delay_iscorrect() {
+	check_if_cache_delay_iscorrect(i64(300 * 1000000), temp_html_n) or { assert false }
+
+	check_if_cache_delay_iscorrect(i64(-100), temp_html_n) or { return }
+	assert false
+}
+
+fn test_cache_request_route() {
+	mut dtmi := init_dtm(false, 0)
+	mut is_cache_exist := true
+	mut cache_delay_expiration := i64(400)
+	mut last_template_mod := get_current_unix_micro_timestamp()
+	mut test_current_template_mod := last_template_mod
+	mut cache_del_exp := 300
+	mut gen_at := last_template_mod
+	mut content_checksum := 'checksumtest1'
+	mut current_content_checksum := 'checksumtest2'
+
+	mut request_type, _ := dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
+		last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
+		get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
+
+	assert request_type == CacheRequest.update
+
+	current_content_checksum = 'checksumtest1'
+
+	request_type, _ = dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
+		last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
+		get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
+
+	assert request_type == CacheRequest.cached
+
+	gen_at = (last_template_mod - 500)
+
+	request_type, _ = dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
+		last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
+		get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
+
+	assert request_type == CacheRequest.exp_update
+
+	is_cache_exist = false
+
+	request_type, _ = dtmi.cache_request_route(is_cache_exist, cache_delay_expiration,
+		last_template_mod, test_current_template_mod, cache_del_exp, gen_at,
+		get_current_unix_micro_timestamp(), content_checksum, current_content_checksum)
+
+	assert request_type == CacheRequest.new
+}
+
+fn testsuite_end() {
+	temp_folder := dtm_test_root_dir()
+	os.rmdir_all(temp_folder) or {}
+}
+
+// Utilities function :
+
+fn init_dtm(b bool, m int) &DynamicTemplateManager {
+	temp_folder := dtm_test_root_dir()
+	templates_path := os.join_path(temp_folder, temp_templates_dir)
+
+	init_params := DynamicTemplateManagerInitialisationParams{
+		active_cache_server:  b
+		max_size_data_in_mem: m
+		test_template_dir:    templates_path
+	}
+
+	dtm := initialize(init_params)
+
+	return dtm
+}

tools/v/vlib/x/templating/dtm/escape_html_strings_in_templates.v

@@ -0,0 +1,30 @@
+module dtm
+
+import strings
+
+fn filter(s string) string {
+	mut escaped := strings.new_builder(s.len)
+	for i := 0; i < s.len; i++ {
+		match s[i] {
+			`&` {
+				escaped.write_string('&amp;')
+			}
+			`<` {
+				escaped.write_string('&lt;')
+			}
+			`>` {
+				escaped.write_string('&gt;')
+			}
+			`"` {
+				escaped.write_string('&#34;')
+			}
+			`'` {
+				escaped.write_string('&#39;')
+			}
+			else {
+				escaped.write_u8(s[i])
+			}
+		}
+	}
+	return escaped.str()
+}

tools/v/vlib/x/templating/dtm/tmpl.v

@@ -0,0 +1,358 @@
+// Copyright (c) 2019-2024 Alexander Medvednikov. All rights reserved.
+// Use of this source code is governed by an MIT license
+// that can be found in the LICENSE file.
+
+/*
+This source code originates from the internal V compiler 'vlib/v/parser/tmpl.v' and
+has been heavily modified for the needs of the Dynamic Template Manager. Thanks to its original author, Alexander Medvednikov.
+*/
+
+module dtm
+
+import os
+import strings
+
+enum State {
+	simple // default - no special interpretation of tags, *at all*!
+	// That is suitable for the general case of text template interpolation,
+	// for example for interpolating arbitrary source code (even V source) templates.
+
+	html // default, only when the template extension is .html
+	css  // <style>
+	js   // <script>
+	// span // span.{
+}
+
+fn (mut state State) update(line string) {
+	trimmed_line := line.trim_space()
+	if is_html_open_tag('style', line) {
+		state = .css
+	} else if trimmed_line == '</style>' {
+		state = .html
+	} else if is_html_open_tag('script', line) {
+		state = .js
+	} else if trimmed_line == '</script>' {
+		state = .html
+	}
+}
+
+const tmpl_str_end = "')\n"
+
+// check HTML open tag `<name attr="x" >`
+fn is_html_open_tag(name string, s string) bool {
+	trimmed_line := s.trim_space()
+	mut len := trimmed_line.len
+
+	if len < name.len {
+		return false
+	}
+
+	mut sub := trimmed_line[0..1]
+	if sub != '<' { // not start with '<'
+		return false
+	}
+	sub = trimmed_line[len - 1..len]
+	if sub != '>' { // not end with '<'
+		return false
+	}
+	sub = trimmed_line[len - 2..len - 1]
+	if sub == '/' { // self-closing
+		return false
+	}
+	sub = trimmed_line[1..len - 1]
+	if sub.contains_any('<>') { // `<name <bad> >`
+		return false
+	}
+	if sub == name { // `<name>`
+		return true
+	} else {
+		len = name.len
+		if sub.len <= len { // `<nam>` or `<meme>`
+			return false
+		}
+		if sub[..len + 1] != '${name} ' { // not `<name ...>`
+			return false
+		}
+		return true
+	}
+}
+
+fn replace_placeholders_with_data(line string, data &map[string]DtmMultiTypeMap, state State) string {
+	mut rline := line
+	if data.len == 0 {
+		return rline
+	}
+	mut need_include_html := false
+
+	for key, value in data {
+		mut placeholder := '$${key}'
+
+		if placeholder.ends_with(include_html_key_tag) {
+			placeholder = placeholder.all_before_last(include_html_key_tag)
+			need_include_html = true
+		}
+		if !rline.contains(placeholder) {
+			need_include_html = false
+			continue
+		}
+		mut val_str := ''
+		match value {
+			i8, i16, int, i64, u8, u16, u32, u64, f32, f64 {
+				// Converts value to string
+				temp_val := value.str()
+				// Filters the string value for safe insertion
+				val_str = filter(temp_val)
+			}
+			string {
+				// Checks if the placeholder allows HTML inclusion
+				if need_include_html {
+					if state == State.html {
+						// Escape the whole value first, then restore only the explicit allow-list.
+						// This preserves the documented opt-in HTML behavior without allowing
+						// arbitrary raw tags through `_#includehtml`.
+						val_str = filter(value)
+						for tag in allowed_tags {
+							escaped_tag := filter(tag)
+							val_str = val_str.replace(escaped_tag, tag)
+						}
+					} else {
+						val_str = filter(value)
+					}
+					need_include_html = false
+				} else {
+					// Filters the string value for safe insertion
+					val_str = filter(value)
+				}
+			}
+		}
+
+		rline = rline.replace(placeholder, val_str)
+	}
+	return rline
+}
+
+fn insert_template_code(fn_name string, tmpl_str_start string, line string, data &map[string]DtmMultiTypeMap,
+	state State) string {
+	// HTML, may include `@var`
+	// escaped by cgen, unless it's a `veb.RawHtml` string
+	trailing_bs := tmpl_str_end + 'sb_${fn_name}.write_u8(92)\n' + tmpl_str_start
+	mut rline := line.replace('\\', '\\\\').replace("'", "\\'").replace('@', '$')
+	rline = rline.replace(r'$$', r'\@').replace(r'.$', r'.@')
+	comptime_call_str := rline.find_between('\${', '}')
+	if comptime_call_str.contains("\\'") {
+		rline = rline.replace(comptime_call_str, comptime_call_str.replace("\\'", r"'"))
+	}
+	if rline.ends_with('\\') {
+		rline = rline[0..rline.len - 2] + trailing_bs
+	}
+	if rline.contains('$') {
+		rline = replace_placeholders_with_data(rline, data, state)
+	}
+	return rline
+}
+
+// compile_file compiles the content of a file by the given path as a template
+fn compile_template_file(template_file string, fn_name string, data &map[string]DtmMultiTypeMap) string {
+	template_content := os.read_file(template_file) or {
+		eprintln('${message_signature_error} Template generator can not reading from ${template_file} file')
+		return internat_server_error
+	}
+	mut lines := template_content.split_into_lines()
+
+	basepath := os.dir(template_file)
+
+	tmpl_str_start := "\tsb_${fn_name}.write_string('"
+	mut source := strings.new_builder(1000)
+
+	mut state := State.simple
+	template_ext := os.file_ext(template_file)
+	if template_ext.to_lower() == '.html' {
+		state = .html
+	}
+
+	mut in_span := false
+	mut end_of_line_pos := 0
+	mut start_of_line_pos := 0
+	mut tline_number := -1 // keep the original line numbers, even after insert/delete ops on lines; `i` changes
+	for i := 0; i < lines.len; i++ {
+		line := lines[i]
+		tline_number++
+		start_of_line_pos = end_of_line_pos
+		end_of_line_pos += line.len + 1
+		if state != .simple {
+			state.update(line)
+		}
+		$if trace_tmpl ? {
+			eprintln('>>> tfile: ${template_file}, spos: ${start_of_line_pos:6}, epos:${end_of_line_pos:6}, fi: ${tline_number:5}, i: ${i:5}, state: ${state:10}, line: ${line}')
+		}
+		if line.contains('@header') {
+			position := line.index('@header') or { 0 }
+			eprintln("${message_signature_warn} Please use @include 'header' instead of @header (deprecated), position : ${position}")
+			continue
+		}
+		if line.contains('@footer') {
+			position := line.index('@footer') or { 0 }
+			eprintln("${message_signature_warn} Please use @include 'footer' instead of @footer (deprecated), position : ${position}")
+			continue
+		}
+		if line.contains('@include ') {
+			lines.delete(i)
+			// Allow single or double quoted paths.
+			mut file_name := if line.contains('"') {
+				line.split('"')[1]
+			} else if line.contains("'") {
+				line.split("'")[1]
+			} else {
+				s := '@include '
+				position := line.index(s) or { 0 }
+				eprintln("${message_signature_error} path for @include must be quoted with ' or \" without line breaks or extraneous characters between @include and the quotes, position : ${position}")
+				return internat_server_error
+			}
+			mut file_ext := os.file_ext(file_name)
+			if file_ext == '' {
+				file_ext = '.html'
+			}
+			file_name = file_name.replace(file_ext, '')
+			// relative path, starting with the current folder
+			mut templates_folder := os.real_path(basepath)
+			if file_name.contains('/') && file_name.starts_with('/') {
+				// an absolute path
+				templates_folder = ''
+			}
+			file_path := os.real_path(os.join_path_single(templates_folder,
+				'${file_name}${file_ext}'))
+			$if trace_tmpl ? {
+				eprintln('>>> basepath: "${basepath}" , template_file: "${template_file}" , fn_name: "${fn_name}" , @include line: "${line}" , file_name: "${file_name}" , file_ext: "${file_ext}" , templates_folder: "${templates_folder}" , file_path: "${file_path}"')
+			}
+			file_content := os.read_file(file_path) or {
+				position := line.index('@include ') or { 0 } + '@include '.len
+				eprintln('${message_signature_error} Reading @include file "${file_name}" from path: ${file_path} failed, position : ${position}')
+				return internat_server_error
+			}
+
+			file_splitted := file_content.split_into_lines().reverse()
+			for f in file_splitted {
+				tline_number--
+				lines.insert(i, f)
+			}
+			i--
+			continue
+		}
+		if line.contains('@if ') {
+			/*			source.writeln(dtm.tmpl_str_end)
+			pos := line.index('@if') or { continue }
+			source.writeln('if ' + line[pos + 4..] + '{')
+			source.writeln(tmpl_str_start) */
+			continue
+		}
+		if line.contains('@end') {
+			/*			// Remove new line byte
+			source.go_back(1)
+			source.writeln(dtm.tmpl_str_end)
+			source.writeln('}')
+			source.writeln(tmpl_str_start) */
+			continue
+		}
+		if line.contains('@else') {
+			// Remove new line byte
+			source.go_back(1)
+			/*			source.writeln(dtm.tmpl_str_end)
+			source.writeln(' } else { ')
+			source.writeln(tmpl_str_start) */
+			continue
+		}
+		if line.contains('@for') {
+			/*			source.writeln(dtm.tmpl_str_end)
+			pos := line.index('@for') or { continue }
+			source.writeln('for ' + line[pos + 4..] + '{')
+			source.writeln(tmpl_str_start) */
+			continue
+		}
+		if state == .simple {
+			// by default, just copy 1:1
+			source.writeln(insert_template_code(fn_name, tmpl_str_start, line, data, state))
+			continue
+		}
+		// The .simple mode ends here. The rest handles .html/.css/.js state transitions.
+
+		if state != .simple {
+			if line.contains('@js ') {
+				pos := line.index('@js') or { continue }
+				source.write_string('<script src="')
+				source.write_string(line[pos + 5..line.len - 1])
+				source.writeln('"></script>')
+				continue
+			}
+			if line.contains('@css ') {
+				pos := line.index('@css') or { continue }
+				source.write_string('<link href="')
+				source.write_string(line[pos + 6..line.len - 1])
+				source.writeln('" rel="stylesheet" type="text/css">')
+				continue
+			}
+		}
+
+		match state {
+			.html {
+				line_t := line.trim_space()
+				if line_t.starts_with('span.') && line.ends_with('{') {
+					//`span.header {` => `<span class='header'>`
+					class := line.find_between('span.', '{').trim_space()
+					source.writeln('<span class="${class}">')
+					in_span = true
+					continue
+				} else if line_t.starts_with('.') && line.ends_with('{') {
+					//`.header {` => `<div class='header'>`
+					class := line.find_between('.', '{').trim_space()
+					trimmed := line.trim_space()
+					source.write_string(strings.repeat(`\t`, line.len - trimmed.len)) // add the necessary indent to keep <div><div><div> code clean
+					source.writeln('<div class="${class}">')
+					continue
+				} else if line_t.starts_with('#') && line.ends_with('{') {
+					//`#header {` => `<div id='header'>`
+					class := line.find_between('#', '{').trim_space()
+					source.writeln('<div id="${class}">')
+					continue
+				} else if line_t == '}' {
+					source.write_string(strings.repeat(`\t`, line.len - line_t.len)) // add the necessary indent to keep <div><div><div> code clean
+					if in_span {
+						source.writeln('</span>')
+						in_span = false
+					} else {
+						source.writeln('</div>')
+					}
+					continue
+				}
+			}
+			.js {
+				// if line.contains('//V_TEMPLATE') {
+				source.writeln(insert_template_code(fn_name, tmpl_str_start, line, data, state))
+				//} else {
+				// replace `$` to `\$` at first to escape JavaScript template literal syntax
+				// source.writeln(line.replace(r'$', r'\$').replace(r'$$', r'@').replace(r'.$',
+				// r'.@').replace(r"'", r"\'"))
+				//}
+				continue
+			}
+			.css {
+				// disable template variable declaration in inline stylesheet
+				// because of  some CSS rules prefixed with `@`.
+				source.writeln(line.replace(r'.$', r'.@').replace(r"'", r"\'"))
+				continue
+			}
+			else {}
+		}
+
+		// by default, just copy 1:1
+		source.writeln(insert_template_code(fn_name, tmpl_str_start, line, data, state))
+	}
+
+	result := source.str()
+	$if trace_tmpl_expansion ? {
+		eprintln('>>>>>>> template expanded to:')
+		eprintln(result)
+		eprintln('-----------------------------')
+	}
+
+	return result
+}