Compare commits

...

10 Commits

Author SHA1 Message Date
Tom Alexander
3428a3f509
Increment the version for publishing to crates.io. 2021-02-06 16:28:56 -05:00
Tom Alexander
0302ed216f
Add my implementation of a take_until_parser_matches parser.
The author of nom is too busy to review the PR, and cargo does not allow for git dependencies, so I am going to copy my implementation into this code base so I can use upstream nom so I can push to cargo. While this code has been submitted upstream to nom which is under the MIT license, I am the author of this code so I believe I have the full right to also release it in this project under the 0BSD license.
2021-02-06 16:23:54 -05:00
Tom Alexander
b4dd4cebfd
Expose the integrations through the library so they can be used outside this project. 2020-12-30 15:17:48 -05:00
Tom Alexander
7cb79f6762
Add metadata for pushing to crates.io 2020-12-29 18:31:44 -05:00
Tom Alexander
bd1f8cb383
Merge branch 'task/separate_out_json_implementation' 2020-12-29 18:22:56 -05:00
Tom Alexander
900d929869
Move the json integration to its own file to keep the serde stuff separate. 2020-12-29 18:21:12 -05:00
Tom Alexander
a9a83d1b4a
Update for the latest nom and make serde an optional dep. 2020-12-29 18:07:49 -05:00
Tom Alexander
aa3ed99fca
Create a json-integration feature flag and disable building the CLI program without it. 2020-12-29 17:51:46 -05:00
Tom Alexander
7602599cf2
Add todo.md to the gitignore. 2020-12-29 17:40:41 -05:00
Tom Alexander
5df2d19212
Updating README. 2020-06-27 16:57:21 -04:00
10 changed files with 870 additions and 712 deletions

3
.gitignore vendored
View File

@ -9,3 +9,6 @@ Cargo.lock
# Javascript junk if you run the compliance tests sans docker
js/node_modules
js/package-lock.json
# I symlink a todo file into this repo
todo.md

View File

@ -1,8 +1,18 @@
[package]
name = "duster"
version = "0.1.0"
version = "0.1.1"
authors = ["Tom Alexander <tom@fizz.buzz>"]
description = "A rust implementation of the dustjs template engine."
edition = "2018"
license = "0BSD"
repository = "https://code.fizz.buzz/talexander/duster"
readme = "README.md"
keywords = ["dust", "dustjs", "templating", "web"]
categories = ["template-engine"]
[features]
default = ["json-integration"]
json-integration = ["serde", "serde_json"]
[lib]
name = "duster"
@ -11,8 +21,11 @@ path = "src/lib.rs"
[[bin]]
name = "duster-cli"
path = "src/bin.rs"
required-features = ["json-integration"]
[dependencies]
nom = { git = "https://github.com/tomalexander/nom.git", branch = "take_until_parser_matches" }
serde = "1.0.106"
serde_json = "1.0.51"
nom = "6.1.0"
# The author of nom is too busy to review the PR, and cargo does not allow for git dependencies, so I am going to copy my implementation into this code base so I can use upstream nom so I can push to cargo.
# nom = { git = "https://github.com/tomalexander/nom.git", branch = "take_until_parser_matches" }
serde = { version = "1.0.106", optional = true }
serde_json = { version = "1.0.51", optional = true }

View File

@ -2,6 +2,27 @@
An implementation of the [LinkedIn fork of DustJS](https://www.dustjs.com/) written in rust.
**NOT RECOMMENDED FOR PUBLIC USE**
**WARNING: Early-stage project**
This code is available free and open source under the [0BSD](https://choosealicense.com/licenses/0bsd/), but it is a very early-stage project. You're welcome to use it, fork it, print it out and fold it into a hat, etc... but you will find that this project is not yet polished nor feature complete. While this repository uses the 0BSD license which does not require the inclusion of a copyright notice/text in any distribution, it depends on [nom](https://github.com/Geal/nom) which is under the MIT license, the Rust standard library which is [dual licensed](https://github.com/rust-lang/rust/issues/67014), serde_json which is [dual licensed](https://github.com/serde-rs/json), and serde which is [dual licensed](https://github.com/serde-rs/serde).
While I've added a lot of tests proving a byte-for-byte compatibility with the official LinkedIn DustJS implementation, this project has not (afaik) been used in any sort of large production environment. If you find any incompatibilities between this implementation and the LinkedIn DustJS implementation, please let me know (or even better: commit a test under `js/test_cases`!).
## Differences between duster and LinkedIn DustJS
### Context Helpers
LinkedIn DustJS supports embedding javascript functions inside the render context itself. I will never be adding a javascript engine to this project, so those functions will not work in duster. I do have two plans to provide similar functionality:
1. In the future I plan to officially support embedding rust functions inside your render context. I believe this functionality is already possible by wrapping your function inside a struct and putting your logic in the render function, but I'm hoping to provide a more standardized way to accomplish this.
2. In the future I plan to attempt to compile this code to web assembly for running on the frontend. I assume this would also allow me to execute javascript using the browser's javascript engine, but I have not looked into this yet.
### Unicode support
The parser combinator library that I am using (nom) does not support unicode characters. In the future I will fix this, either by writing new parsers for the nom framework or by writing my own parser combinator framework. I'd prefer the former option since using a widely used parser combinator framework allows for greater interoperability across other projects (for example, if a nom-based html parser wanted to add support for parsing dust templates inside an `text/x-dust-template` tag, they could simply embed my parser inside theirs).
## Dependencies
While this repository uses the [0BSD license](https://choosealicense.com/licenses/0bsd/) which does not require the inclusion of a copyright notice/text in any distribution, it depends on some bsd-style licensed libraries including (but potentially not limited to):
- the Rust standard library which is [dual licensed](https://github.com/rust-lang/rust/issues/67014).
- [nom](https://github.com/Geal/nom) which is under the MIT license.
- [serde](https://github.com/serde-rs/serde) which is dual licensed.
- [serde_json](https://github.com/serde-rs/json) which is dual licensed but only used in the binary, not in the library
I am not a lawyer, and I am not your lawyer. This is not legal advice, but I believe attribution for these projects and their dependencies would be required under the terms of their licenses.

View File

@ -1,33 +1,15 @@
extern crate nom;
use crate::renderer::CompareContextElement;
use parser::Filter;
use parser::OwnedLiteral;
use parser::Template;
use renderer::compare_json_numbers;
use renderer::compile_template;
use renderer::Castable;
use renderer::ComparisonNumber;
use renderer::CompileError;
use renderer::ContextElement;
use renderer::DustRenderer;
use renderer::IceResult;
use renderer::IntoContextElement;
use renderer::Loopable;
use renderer::MathNumber;
use renderer::RenderError;
use renderer::Renderable;
use renderer::Sizable;
use renderer::Truthiness;
use renderer::WalkError;
use renderer::Walkable;
use std::cmp::Ordering;
use std::convert::TryInto;
use std::env;
use std::fs;
use std::io::{self, Read};
use std::path::Path;
mod integrations;
mod parser;
mod renderer;
@ -90,680 +72,3 @@ fn read_context_from_stdin() -> serde_json::Value {
serde_json::from_str(&buffer).expect("Failed to parse json")
}
fn html_escape(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'<' => output.push_str("&lt;"),
'>' => output.push_str("&gt;"),
'"' => output.push_str("&quot;"),
'\'' => output.push_str("&#39;"),
'&' => output.push_str("&amp;"),
_ => output.push(c),
});
output
}
fn javascript_escape(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'"' => output.push_str(r#"\""#),
'\'' => output.push_str(r#"\'"#),
'\t' => output.push_str(r#"\t"#),
'\x0C' => output.push_str(r#"\f"#),
'\n' => output.push_str(r#"\n"#),
'\r' => output.push_str(r#"\r"#),
'\\' => output.push_str(r#"\\"#),
'/' => output.push_str(r#"\/"#),
_ => output.push(c),
});
output
}
fn get_utf8_hex(inp: char) -> String {
let num_bytes = inp.len_utf8();
let mut byte_buffer = [0; 4]; // UTF-8 supports up to 4 bytes per codepoint
let mut output = String::with_capacity(num_bytes * 2);
inp.encode_utf8(&mut byte_buffer);
for b in &byte_buffer[..num_bytes] {
output.push_str(&format!("{:02X}", b));
}
output
}
fn encode_uri(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' | 'a' | 'b' | 'c' | 'd' | 'e'
| 'f' | 'g' | 'h' | 'i' | 'j' | 'k' | 'l' | 'm' | 'n' | 'o' | 'p' | 'q' | 'r' | 's'
| 't' | 'u' | 'v' | 'w' | 'x' | 'y' | 'z' | 'A' | 'B' | 'C' | 'D' | 'E' | 'F' | 'G'
| 'H' | 'I' | 'J' | 'K' | 'L' | 'M' | 'N' | 'O' | 'P' | 'Q' | 'R' | 'S' | 'T' | 'U'
| 'V' | 'W' | 'X' | 'Y' | 'Z' | ';' | ',' | '/' | '?' | ':' | '@' | '&' | '=' | '+'
| '$' | '-' | '_' | '.' | '!' | '~' | '*' | '\'' | '(' | ')' | '#' => output.push(c),
_ => {
output.push('%');
output.push_str(&get_utf8_hex(c));
}
});
output
}
fn encode_uri_component(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' | 'a' | 'b' | 'c' | 'd' | 'e'
| 'f' | 'g' | 'h' | 'i' | 'j' | 'k' | 'l' | 'm' | 'n' | 'o' | 'p' | 'q' | 'r' | 's'
| 't' | 'u' | 'v' | 'w' | 'x' | 'y' | 'z' | 'A' | 'B' | 'C' | 'D' | 'E' | 'F' | 'G'
| 'H' | 'I' | 'J' | 'K' | 'L' | 'M' | 'N' | 'O' | 'P' | 'Q' | 'R' | 'S' | 'T' | 'U'
| 'V' | 'W' | 'X' | 'Y' | 'Z' | '-' | '_' | '.' | '!' | '~' | '*' | '\'' | '(' | ')' => {
output.push(c)
}
_ => {
output.push('%');
output.push_str(&get_utf8_hex(c));
}
});
output
}
fn apply_filter(
json_value: &serde_json::Value,
filter: &Filter,
) -> Result<serde_json::Value, RenderError> {
match (json_value, filter) {
// Html escape filter
(serde_json::Value::String(string), Filter::HtmlEncode) => {
Ok(serde_json::Value::String(html_escape(string)))
}
(_, Filter::HtmlEncode) => Ok(serde_json::Value::String(html_escape(
&json_value.render(&Vec::new())?,
))),
// Disable html escape filter
(_, Filter::DisableHtmlEncode) => panic!("The |s filter is automatically removed by the renderer since it is a no-op during rendering."),
// Parse JSON filter
(serde_json::Value::String(string), Filter::JsonParse) => {
serde_json::from_str(&string).or(Err(RenderError::InvalidJson(string.to_owned())))
}
(_, Filter::JsonParse) => {
let rendered_value = json_value.render(&Vec::new())?;
serde_json::from_str(&rendered_value).or(Err(RenderError::InvalidJson(rendered_value)))
}
// Json Stringify filter
(_, Filter::JsonStringify) => {
Ok(serde_json::Value::String(json_value.to_string()))
}
// Javascript escape filter
(serde_json::Value::String(string), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::String(javascript_escape(string)))
}
(serde_json::Value::Bool(boolean), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Bool(*boolean))
}
(serde_json::Value::Number(number), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Number(number.clone()))
}
(serde_json::Value::Array(arr), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Array(arr.clone()))
}
(serde_json::Value::Object(obj), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Object(obj.clone()))
}
(_, Filter::JavascriptStringEncode) => Ok(serde_json::Value::String(javascript_escape(
&json_value.render(&Vec::new())?,
))),
// EncodeURI filter
(serde_json::Value::String(string), Filter::EncodeUri) => {
Ok(serde_json::Value::String(encode_uri(string)))
}
(_, Filter::EncodeUri) => Ok(serde_json::Value::String(encode_uri(
&json_value.render(&Vec::new())?,
))),
// EncodeURIComponent filter
(serde_json::Value::String(string), Filter::EncodeUriComponent) => {
Ok(serde_json::Value::String(encode_uri_component(string)))
}
(_, Filter::EncodeUriComponent) => Ok(serde_json::Value::String(encode_uri_component(
&json_value.render(&Vec::new())?,
))),
}
}
fn apply_filters(
json_value: &serde_json::Value,
filters: &[Filter],
) -> Result<serde_json::Value, RenderError> {
let mut final_value: serde_json::Value = apply_filter(json_value, &filters[0])?;
for filter in &filters[1..] {
final_value = apply_filter(&final_value, filter)?;
}
Ok(final_value)
}
impl ContextElement for serde_json::Value {}
impl Truthiness for serde_json::Value {
fn is_truthy(&self) -> bool {
match self {
serde_json::Value::Null => false,
serde_json::Value::Bool(boolean) => *boolean,
serde_json::Value::Number(_num) => true,
serde_json::Value::String(string_value) => !string_value.is_empty(),
serde_json::Value::Array(array_value) => !array_value.is_empty(),
serde_json::Value::Object(_obj) => true,
}
}
}
impl Renderable for serde_json::Value {
fn render(&self, _filters: &Vec<Filter>) -> Result<String, RenderError> {
let after_apply = if _filters.is_empty() {
None
} else {
Some(apply_filters(self, _filters)?)
};
match after_apply.as_ref().unwrap_or(self) {
serde_json::Value::Null => Ok("".to_owned()),
serde_json::Value::Bool(boolean) => Ok(boolean.to_string()),
serde_json::Value::Number(num) => Ok(num.to_string()),
serde_json::Value::String(string) => Ok(string.to_string()),
serde_json::Value::Array(arr) => {
let rendered: Result<Vec<String>, RenderError> =
arr.iter().map(|val| val.render(&Vec::new())).collect();
let rendered_slice: &[String] = &rendered?;
Ok(rendered_slice.join(","))
}
serde_json::Value::Object(_obj) => Ok("[object Object]".to_owned()),
}
}
}
impl Walkable for serde_json::Value {
fn walk(&self, segment: &str) -> Result<&dyn IntoContextElement, WalkError> {
match self {
serde_json::Value::Null => Err(WalkError::CantWalk),
serde_json::Value::Bool(_boolean) => Err(WalkError::CantWalk),
serde_json::Value::Number(_num) => Err(WalkError::CantWalk),
serde_json::Value::String(_string) => Err(WalkError::CantWalk),
serde_json::Value::Array(_arr) => Err(WalkError::CantWalk),
serde_json::Value::Object(obj) => obj
.get(segment)
.map(|val| val as _)
.ok_or(WalkError::CantWalk),
}
}
}
impl Loopable for serde_json::Value {
fn get_loop_elements(&self) -> Vec<&dyn ContextElement> {
match self {
serde_json::Value::Array(array_value) => array_value.iter().map(|x| x as _).collect(),
_ => Vec::new(),
}
}
}
impl Sizable for serde_json::Value {
fn is_castable(&self) -> bool {
match self {
serde_json::Value::Null => true,
serde_json::Value::Bool(_) => false,
serde_json::Value::Number(_) => true,
serde_json::Value::String(_) => true,
serde_json::Value::Array(_) => true,
serde_json::Value::Object(_) => true,
}
}
fn get_size<'a>(&'a self) -> Option<IceResult<'a>> {
match self {
serde_json::Value::Null => {
Some(IceResult::from_owned(OwnedLiteral::LPositiveInteger(0)))
}
serde_json::Value::Bool(_boolean) => {
Some(IceResult::from_owned(OwnedLiteral::LPositiveInteger(0)))
}
serde_json::Value::Number(_num) => Some(IceResult::from_borrowed(self)),
serde_json::Value::String(text) => Some(IceResult::from_owned(
OwnedLiteral::LPositiveInteger(text.len().try_into().unwrap()),
)),
serde_json::Value::Array(arr) => Some(IceResult::from_owned(
OwnedLiteral::LPositiveInteger(arr.len().try_into().unwrap()),
)),
serde_json::Value::Object(obj) => Some(IceResult::from_owned(
OwnedLiteral::LPositiveInteger(obj.len().try_into().unwrap()),
)),
}
}
}
impl Castable for serde_json::Value {
fn cast_to_type<'a>(&'a self, target: &str) -> Option<IceResult<'a>> {
match (self, target) {
(serde_json::Value::String(text), "number") => text
.parse::<u64>()
.map(|num| IceResult::from_owned(OwnedLiteral::LPositiveInteger(num)))
.or_else(|_| {
text.parse::<i64>()
.map(|num| IceResult::from_owned(OwnedLiteral::LNegativeInteger(num)))
})
.or_else(|_| {
text.parse::<f64>()
.map(|num| IceResult::from_owned(OwnedLiteral::LFloat(num)))
})
.ok(),
(serde_json::Value::Number(_), "number") => Some(IceResult::from_borrowed(self)),
(serde_json::Value::Null, "number") => {
Some(IceResult::from_owned(serde_json::Value::Number(0.into())))
}
(serde_json::Value::Bool(boolean), "number") => {
if *boolean {
Some(IceResult::from_owned(serde_json::Value::Number(1.into())))
} else {
Some(IceResult::from_owned(serde_json::Value::Number(0.into())))
}
}
(serde_json::Value::Array(_), "number") => None,
(serde_json::Value::Object(_), "number") => None,
(serde_json::Value::String(_), "string") => Some(IceResult::from_borrowed(self)),
(serde_json::Value::Number(num), "string") => Some(IceResult::from_owned(
serde_json::Value::String(num.to_string()),
)),
(serde_json::Value::Null, "string") => Some(IceResult::from_owned(
serde_json::Value::String("null".to_owned()),
)),
(serde_json::Value::Bool(boolean), "string") => Some(IceResult::from_owned(
serde_json::Value::String(boolean.to_string()),
)),
(serde_json::Value::Array(_), "string") => Some(IceResult::from_owned(
serde_json::Value::String(self.render(&Vec::new()).unwrap_or("".to_owned())),
)),
(serde_json::Value::Object(_), "string") => Some(IceResult::from_owned(
serde_json::Value::String(self.render(&Vec::new()).unwrap_or("".to_owned())),
)),
(serde_json::Value::String(text), "boolean") => {
if text.is_empty() {
Some(IceResult::from_owned(serde_json::Value::Bool(false)))
} else {
Some(IceResult::from_owned(serde_json::Value::Bool(true)))
}
}
(serde_json::Value::Number(json_num), "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(
match (json_num.as_u64(), json_num.as_i64(), json_num.as_f64()) {
(Some(num), _, _) => num != 0,
(_, Some(num), _) => num != 0,
(_, _, Some(num)) => num != 0.0 && !num.is_nan(),
_ => false,
},
)))
}
(serde_json::Value::Null, "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(false)))
}
(serde_json::Value::Bool(_), "boolean") => Some(IceResult::from_borrowed(self)),
(serde_json::Value::Array(_), "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(true)))
}
(serde_json::Value::Object(_), "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(true)))
}
(_, _) => panic!("Unimplemented cast"),
}
}
}
impl CompareContextElement for serde_json::Value {
fn equals(&self, other: &dyn ContextElement) -> bool {
// println!("Json equality check {:?} == {:?}", self, other);
// Handle other serde_json::Value
match other.to_any().downcast_ref::<Self>() {
None => (),
Some(other_json_value) => match (self, other_json_value) {
// Non-scalar values not caught in the renderer by the
// identical-path shortcut are always not equal.
(serde_json::Value::Array(_), _)
| (_, serde_json::Value::Array(_))
| (serde_json::Value::Object(_), _)
| (_, serde_json::Value::Object(_)) => return false,
_ => return self == other_json_value,
},
}
// Handle literals
match other.to_any().downcast_ref::<OwnedLiteral>() {
None => (),
Some(OwnedLiteral::LString(other_string)) => {
return self.as_str().map_or(false, |s| s == other_string)
}
Some(OwnedLiteral::LBoolean(boolean)) => {
return self.equals(&serde_json::Value::Bool(*boolean) as &dyn ContextElement);
}
Some(OwnedLiteral::LPositiveInteger(other_num)) => {
let other_json_num: serde_json::Number = std::convert::From::from(*other_num);
return self
.equals(&serde_json::Value::Number(other_json_num) as &dyn ContextElement);
}
Some(OwnedLiteral::LNegativeInteger(other_num)) => {
let other_json_num: serde_json::Number = std::convert::From::from(*other_num);
return self
.equals(&serde_json::Value::Number(other_json_num) as &dyn ContextElement);
}
Some(OwnedLiteral::LFloat(other_num)) => match self.as_f64() {
None => return false,
Some(self_float) => return self_float == *other_num,
},
}
false
}
fn partial_compare(&self, other: &dyn ContextElement) -> Option<Ordering> {
// Handle type coerced objects
// When doing a greater than or less than comparison,
// javascript coerces objects into "[object Object]".
if let serde_json::Value::Object(_) = self {
return OwnedLiteral::LString(self.render(&Vec::new()).unwrap_or("".to_owned()))
.partial_compare(other);
}
// When doing a greater than or less than comparison
// javascript turns arrays into strings.
if let serde_json::Value::Array(_) = self {
return OwnedLiteral::LString(self.render(&Vec::new()).unwrap_or("".to_owned()))
.partial_compare(other);
}
let maybe_json_other = other.to_any().downcast_ref::<Self>();
let maybe_literal_other = other.to_any().downcast_ref::<OwnedLiteral>();
// If they're both strings, compare them directly
match (self, maybe_json_other, maybe_literal_other) {
// If they're both strings, compare them directly
(
serde_json::Value::String(self_string),
Some(serde_json::Value::String(other_string)),
_,
) => return self_string.partial_cmp(&other_string),
(
serde_json::Value::String(self_string),
_,
Some(OwnedLiteral::LString(other_string)),
) => return self_string.partial_cmp(&other_string),
// Otherwise, convert to numbers and compare them that way
(_, Some(json_other), _) => return compare_json_numbers(self, json_other),
(_, _, Some(literal_other)) => return compare_json_numbers(self, literal_other),
_ => panic!("Unimplemented comparison type."),
}
}
fn math_add<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
+ std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
+ std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_subtract<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
- std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
- std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_multiply<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
* std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
* std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_divide<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
/ std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
/ std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_modulus<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
% std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
% std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_abs<'a>(&self) -> Option<IceResult<'a>> {
std::convert::Into::<MathNumber>::into(self)
.math_abs()
.map(IceResult::from_owned)
}
fn math_floor<'a>(&self) -> Option<IceResult<'a>> {
std::convert::Into::<MathNumber>::into(self)
.math_floor()
.map(IceResult::from_owned)
}
fn math_ceil<'a>(&self) -> Option<IceResult<'a>> {
std::convert::Into::<MathNumber>::into(self)
.math_ceil()
.map(IceResult::from_owned)
}
}
impl From<&serde_json::Value> for ComparisonNumber {
/// Convert from a JSON value to a number for comparison based on
/// the logic described at
/// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Greater_than
fn from(original: &serde_json::Value) -> Self {
match original {
serde_json::Value::Null => ComparisonNumber::UnsignedInteger(0),
serde_json::Value::Bool(boolean) => {
if *boolean {
ComparisonNumber::UnsignedInteger(1)
} else {
ComparisonNumber::UnsignedInteger(0)
}
}
serde_json::Value::Number(num) => num.into(),
serde_json::Value::String(text) => text.into(),
serde_json::Value::Array(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
serde_json::Value::Object(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
}
}
}
impl From<&serde_json::Number> for ComparisonNumber {
fn from(original: &serde_json::Number) -> Self {
match original.as_u64() {
Some(num) => return ComparisonNumber::UnsignedInteger(num),
None => (),
};
match original.as_i64() {
Some(num) => return ComparisonNumber::SignedInteger(num),
None => (),
};
match original.as_f64() {
Some(num) => return ComparisonNumber::Decimal(num),
None => (),
};
ComparisonNumber::Failure
}
}
impl From<&serde_json::Value> for MathNumber {
fn from(original: &serde_json::Value) -> Self {
match original {
serde_json::Value::Null => MathNumber::Integer(0),
serde_json::Value::Bool(boolean) => {
if *boolean {
MathNumber::Integer(1)
} else {
MathNumber::Integer(0)
}
}
serde_json::Value::Number(num) => num.into(),
serde_json::Value::String(_) => {
panic!("Strings should not be cast to numbers for math")
}
serde_json::Value::Array(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
serde_json::Value::Object(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
}
}
}
impl From<&serde_json::Number> for MathNumber {
fn from(original: &serde_json::Number) -> Self {
match original.as_u64() {
Some(num) => return MathNumber::Integer(num.try_into().unwrap()),
None => (),
};
match original.as_i64() {
Some(num) => return MathNumber::Integer(num.into()),
None => (),
};
match original.as_f64() {
Some(num) => return MathNumber::Decimal(num),
None => (),
};
MathNumber::Failure
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_nested_array_render() {
let x: serde_json::Value =
serde_json::from_str(r#"[3,5,[7,9]]"#).expect("Failed to parse json");
assert_eq!(
x.render(&Vec::new()),
Ok::<_, RenderError>("3,5,7,9".to_owned())
);
}
#[test]
fn test_html_escape() {
assert_eq!(html_escape("<>&\"'"), "&lt;&gt;&amp;&quot;&#39;".to_owned())
}
}

706
src/integrations/json.rs Normal file
View File

@ -0,0 +1,706 @@
//! This file contains an integration for duster that implements
//! support for using serde_json values for the render context. This
//! is in its own separate file to avoid requiring serde as a
//! dependency since ContextElement can be implemented for any
//! type. Disable the json-integration feature to avoid compiling this
//! file and adding the serde and serde_json dependencies.
//!
//! In order to recreate perfect compatibility with the original
//! dustjs implementation, the logic below needs to follow the "rules"
//! of javascript logic.
use crate::parser::Filter;
use crate::parser::OwnedLiteral;
use crate::renderer::compare_json_numbers;
use crate::renderer::Castable;
use crate::renderer::CompareContextElement;
use crate::renderer::ComparisonNumber;
use crate::renderer::ContextElement;
use crate::renderer::IceResult;
use crate::renderer::IntoContextElement;
use crate::renderer::Loopable;
use crate::renderer::MathNumber;
use crate::renderer::RenderError;
use crate::renderer::Renderable;
use crate::renderer::Sizable;
use crate::renderer::Truthiness;
use crate::renderer::WalkError;
use crate::renderer::Walkable;
use std::cmp::Ordering;
use std::convert::TryInto;
fn html_escape(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'<' => output.push_str("&lt;"),
'>' => output.push_str("&gt;"),
'"' => output.push_str("&quot;"),
'\'' => output.push_str("&#39;"),
'&' => output.push_str("&amp;"),
_ => output.push(c),
});
output
}
fn javascript_escape(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'"' => output.push_str(r#"\""#),
'\'' => output.push_str(r#"\'"#),
'\t' => output.push_str(r#"\t"#),
'\x0C' => output.push_str(r#"\f"#),
'\n' => output.push_str(r#"\n"#),
'\r' => output.push_str(r#"\r"#),
'\\' => output.push_str(r#"\\"#),
'/' => output.push_str(r#"\/"#),
_ => output.push(c),
});
output
}
fn get_utf8_hex(inp: char) -> String {
let num_bytes = inp.len_utf8();
let mut byte_buffer = [0; 4]; // UTF-8 supports up to 4 bytes per codepoint
let mut output = String::with_capacity(num_bytes * 2);
inp.encode_utf8(&mut byte_buffer);
for b in &byte_buffer[..num_bytes] {
output.push_str(&format!("{:02X}", b));
}
output
}
fn encode_uri(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' | 'a' | 'b' | 'c' | 'd' | 'e'
| 'f' | 'g' | 'h' | 'i' | 'j' | 'k' | 'l' | 'm' | 'n' | 'o' | 'p' | 'q' | 'r' | 's'
| 't' | 'u' | 'v' | 'w' | 'x' | 'y' | 'z' | 'A' | 'B' | 'C' | 'D' | 'E' | 'F' | 'G'
| 'H' | 'I' | 'J' | 'K' | 'L' | 'M' | 'N' | 'O' | 'P' | 'Q' | 'R' | 'S' | 'T' | 'U'
| 'V' | 'W' | 'X' | 'Y' | 'Z' | ';' | ',' | '/' | '?' | ':' | '@' | '&' | '=' | '+'
| '$' | '-' | '_' | '.' | '!' | '~' | '*' | '\'' | '(' | ')' | '#' => output.push(c),
_ => {
output.push('%');
output.push_str(&get_utf8_hex(c));
}
});
output
}
fn encode_uri_component(inp: &str) -> String {
// Adding 10% space from the original to avoid re-allocations by
// leaving room for escaped sequences.
let mut output = String::with_capacity(((inp.len() as f64) * 1.1) as usize);
inp.chars().for_each(|c| match c {
'0' | '1' | '2' | '3' | '4' | '5' | '6' | '7' | '8' | '9' | 'a' | 'b' | 'c' | 'd' | 'e'
| 'f' | 'g' | 'h' | 'i' | 'j' | 'k' | 'l' | 'm' | 'n' | 'o' | 'p' | 'q' | 'r' | 's'
| 't' | 'u' | 'v' | 'w' | 'x' | 'y' | 'z' | 'A' | 'B' | 'C' | 'D' | 'E' | 'F' | 'G'
| 'H' | 'I' | 'J' | 'K' | 'L' | 'M' | 'N' | 'O' | 'P' | 'Q' | 'R' | 'S' | 'T' | 'U'
| 'V' | 'W' | 'X' | 'Y' | 'Z' | '-' | '_' | '.' | '!' | '~' | '*' | '\'' | '(' | ')' => {
output.push(c)
}
_ => {
output.push('%');
output.push_str(&get_utf8_hex(c));
}
});
output
}
fn apply_filter(
json_value: &serde_json::Value,
filter: &Filter,
) -> Result<serde_json::Value, RenderError> {
match (json_value, filter) {
// Html escape filter
(serde_json::Value::String(string), Filter::HtmlEncode) => {
Ok(serde_json::Value::String(html_escape(string)))
}
(_, Filter::HtmlEncode) => Ok(serde_json::Value::String(html_escape(
&json_value.render(&Vec::new())?,
))),
// Disable html escape filter
(_, Filter::DisableHtmlEncode) => panic!("The |s filter is automatically removed by the renderer since it is a no-op during rendering."),
// Parse JSON filter
(serde_json::Value::String(string), Filter::JsonParse) => {
serde_json::from_str(&string).or(Err(RenderError::InvalidJson(string.to_owned())))
}
(_, Filter::JsonParse) => {
let rendered_value = json_value.render(&Vec::new())?;
serde_json::from_str(&rendered_value).or(Err(RenderError::InvalidJson(rendered_value)))
}
// Json Stringify filter
(_, Filter::JsonStringify) => {
Ok(serde_json::Value::String(json_value.to_string()))
}
// Javascript escape filter
(serde_json::Value::String(string), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::String(javascript_escape(string)))
}
(serde_json::Value::Bool(boolean), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Bool(*boolean))
}
(serde_json::Value::Number(number), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Number(number.clone()))
}
(serde_json::Value::Array(arr), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Array(arr.clone()))
}
(serde_json::Value::Object(obj), Filter::JavascriptStringEncode) => {
Ok(serde_json::Value::Object(obj.clone()))
}
(_, Filter::JavascriptStringEncode) => Ok(serde_json::Value::String(javascript_escape(
&json_value.render(&Vec::new())?,
))),
// EncodeURI filter
(serde_json::Value::String(string), Filter::EncodeUri) => {
Ok(serde_json::Value::String(encode_uri(string)))
}
(_, Filter::EncodeUri) => Ok(serde_json::Value::String(encode_uri(
&json_value.render(&Vec::new())?,
))),
// EncodeURIComponent filter
(serde_json::Value::String(string), Filter::EncodeUriComponent) => {
Ok(serde_json::Value::String(encode_uri_component(string)))
}
(_, Filter::EncodeUriComponent) => Ok(serde_json::Value::String(encode_uri_component(
&json_value.render(&Vec::new())?,
))),
}
}
fn apply_filters(
json_value: &serde_json::Value,
filters: &[Filter],
) -> Result<serde_json::Value, RenderError> {
let mut final_value: serde_json::Value = apply_filter(json_value, &filters[0])?;
for filter in &filters[1..] {
final_value = apply_filter(&final_value, filter)?;
}
Ok(final_value)
}
impl ContextElement for serde_json::Value {}
impl Truthiness for serde_json::Value {
fn is_truthy(&self) -> bool {
match self {
serde_json::Value::Null => false,
serde_json::Value::Bool(boolean) => *boolean,
serde_json::Value::Number(_num) => true,
serde_json::Value::String(string_value) => !string_value.is_empty(),
serde_json::Value::Array(array_value) => !array_value.is_empty(),
serde_json::Value::Object(_obj) => true,
}
}
}
impl Renderable for serde_json::Value {
fn render(&self, _filters: &Vec<Filter>) -> Result<String, RenderError> {
let after_apply = if _filters.is_empty() {
None
} else {
Some(apply_filters(self, _filters)?)
};
match after_apply.as_ref().unwrap_or(self) {
serde_json::Value::Null => Ok("".to_owned()),
serde_json::Value::Bool(boolean) => Ok(boolean.to_string()),
serde_json::Value::Number(num) => Ok(num.to_string()),
serde_json::Value::String(string) => Ok(string.to_string()),
serde_json::Value::Array(arr) => {
let rendered: Result<Vec<String>, RenderError> =
arr.iter().map(|val| val.render(&Vec::new())).collect();
let rendered_slice: &[String] = &rendered?;
Ok(rendered_slice.join(","))
}
serde_json::Value::Object(_obj) => Ok("[object Object]".to_owned()),
}
}
}
impl Walkable for serde_json::Value {
fn walk(&self, segment: &str) -> Result<&dyn IntoContextElement, WalkError> {
match self {
serde_json::Value::Null => Err(WalkError::CantWalk),
serde_json::Value::Bool(_boolean) => Err(WalkError::CantWalk),
serde_json::Value::Number(_num) => Err(WalkError::CantWalk),
serde_json::Value::String(_string) => Err(WalkError::CantWalk),
serde_json::Value::Array(_arr) => Err(WalkError::CantWalk),
serde_json::Value::Object(obj) => obj
.get(segment)
.map(|val| val as _)
.ok_or(WalkError::CantWalk),
}
}
}
impl Loopable for serde_json::Value {
fn get_loop_elements(&self) -> Vec<&dyn ContextElement> {
match self {
serde_json::Value::Array(array_value) => array_value.iter().map(|x| x as _).collect(),
_ => Vec::new(),
}
}
}
impl Sizable for serde_json::Value {
fn is_castable(&self) -> bool {
match self {
serde_json::Value::Null => true,
serde_json::Value::Bool(_) => false,
serde_json::Value::Number(_) => true,
serde_json::Value::String(_) => true,
serde_json::Value::Array(_) => true,
serde_json::Value::Object(_) => true,
}
}
fn get_size<'a>(&'a self) -> Option<IceResult<'a>> {
match self {
serde_json::Value::Null => {
Some(IceResult::from_owned(OwnedLiteral::LPositiveInteger(0)))
}
serde_json::Value::Bool(_boolean) => {
Some(IceResult::from_owned(OwnedLiteral::LPositiveInteger(0)))
}
serde_json::Value::Number(_num) => Some(IceResult::from_borrowed(self)),
serde_json::Value::String(text) => Some(IceResult::from_owned(
OwnedLiteral::LPositiveInteger(text.len().try_into().unwrap()),
)),
serde_json::Value::Array(arr) => Some(IceResult::from_owned(
OwnedLiteral::LPositiveInteger(arr.len().try_into().unwrap()),
)),
serde_json::Value::Object(obj) => Some(IceResult::from_owned(
OwnedLiteral::LPositiveInteger(obj.len().try_into().unwrap()),
)),
}
}
}
impl Castable for serde_json::Value {
fn cast_to_type<'a>(&'a self, target: &str) -> Option<IceResult<'a>> {
match (self, target) {
(serde_json::Value::String(text), "number") => text
.parse::<u64>()
.map(|num| IceResult::from_owned(OwnedLiteral::LPositiveInteger(num)))
.or_else(|_| {
text.parse::<i64>()
.map(|num| IceResult::from_owned(OwnedLiteral::LNegativeInteger(num)))
})
.or_else(|_| {
text.parse::<f64>()
.map(|num| IceResult::from_owned(OwnedLiteral::LFloat(num)))
})
.ok(),
(serde_json::Value::Number(_), "number") => Some(IceResult::from_borrowed(self)),
(serde_json::Value::Null, "number") => {
Some(IceResult::from_owned(serde_json::Value::Number(0.into())))
}
(serde_json::Value::Bool(boolean), "number") => {
if *boolean {
Some(IceResult::from_owned(serde_json::Value::Number(1.into())))
} else {
Some(IceResult::from_owned(serde_json::Value::Number(0.into())))
}
}
(serde_json::Value::Array(_), "number") => None,
(serde_json::Value::Object(_), "number") => None,
(serde_json::Value::String(_), "string") => Some(IceResult::from_borrowed(self)),
(serde_json::Value::Number(num), "string") => Some(IceResult::from_owned(
serde_json::Value::String(num.to_string()),
)),
(serde_json::Value::Null, "string") => Some(IceResult::from_owned(
serde_json::Value::String("null".to_owned()),
)),
(serde_json::Value::Bool(boolean), "string") => Some(IceResult::from_owned(
serde_json::Value::String(boolean.to_string()),
)),
(serde_json::Value::Array(_), "string") => Some(IceResult::from_owned(
serde_json::Value::String(self.render(&Vec::new()).unwrap_or("".to_owned())),
)),
(serde_json::Value::Object(_), "string") => Some(IceResult::from_owned(
serde_json::Value::String(self.render(&Vec::new()).unwrap_or("".to_owned())),
)),
(serde_json::Value::String(text), "boolean") => {
if text.is_empty() {
Some(IceResult::from_owned(serde_json::Value::Bool(false)))
} else {
Some(IceResult::from_owned(serde_json::Value::Bool(true)))
}
}
(serde_json::Value::Number(json_num), "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(
match (json_num.as_u64(), json_num.as_i64(), json_num.as_f64()) {
(Some(num), _, _) => num != 0,
(_, Some(num), _) => num != 0,
(_, _, Some(num)) => num != 0.0 && !num.is_nan(),
_ => false,
},
)))
}
(serde_json::Value::Null, "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(false)))
}
(serde_json::Value::Bool(_), "boolean") => Some(IceResult::from_borrowed(self)),
(serde_json::Value::Array(_), "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(true)))
}
(serde_json::Value::Object(_), "boolean") => {
Some(IceResult::from_owned(serde_json::Value::Bool(true)))
}
(_, _) => panic!("Unimplemented cast"),
}
}
}
impl CompareContextElement for serde_json::Value {
fn equals(&self, other: &dyn ContextElement) -> bool {
// println!("Json equality check {:?} == {:?}", self, other);
// Handle other serde_json::Value
match other.to_any().downcast_ref::<Self>() {
None => (),
Some(other_json_value) => match (self, other_json_value) {
// Non-scalar values not caught in the renderer by the
// identical-path shortcut are always not equal.
(serde_json::Value::Array(_), _)
| (_, serde_json::Value::Array(_))
| (serde_json::Value::Object(_), _)
| (_, serde_json::Value::Object(_)) => return false,
_ => return self == other_json_value,
},
}
// Handle literals
match other.to_any().downcast_ref::<OwnedLiteral>() {
None => (),
Some(OwnedLiteral::LString(other_string)) => {
return self.as_str().map_or(false, |s| s == other_string)
}
Some(OwnedLiteral::LBoolean(boolean)) => {
return self.equals(&serde_json::Value::Bool(*boolean) as &dyn ContextElement);
}
Some(OwnedLiteral::LPositiveInteger(other_num)) => {
let other_json_num: serde_json::Number = std::convert::From::from(*other_num);
return self
.equals(&serde_json::Value::Number(other_json_num) as &dyn ContextElement);
}
Some(OwnedLiteral::LNegativeInteger(other_num)) => {
let other_json_num: serde_json::Number = std::convert::From::from(*other_num);
return self
.equals(&serde_json::Value::Number(other_json_num) as &dyn ContextElement);
}
Some(OwnedLiteral::LFloat(other_num)) => match self.as_f64() {
None => return false,
Some(self_float) => return self_float == *other_num,
},
}
false
}
fn partial_compare(&self, other: &dyn ContextElement) -> Option<Ordering> {
// Handle type coerced objects
// When doing a greater than or less than comparison,
// javascript coerces objects into "[object Object]".
if let serde_json::Value::Object(_) = self {
return OwnedLiteral::LString(self.render(&Vec::new()).unwrap_or("".to_owned()))
.partial_compare(other);
}
// When doing a greater than or less than comparison
// javascript turns arrays into strings.
if let serde_json::Value::Array(_) = self {
return OwnedLiteral::LString(self.render(&Vec::new()).unwrap_or("".to_owned()))
.partial_compare(other);
}
let maybe_json_other = other.to_any().downcast_ref::<Self>();
let maybe_literal_other = other.to_any().downcast_ref::<OwnedLiteral>();
// If they're both strings, compare them directly
match (self, maybe_json_other, maybe_literal_other) {
// If they're both strings, compare them directly
(
serde_json::Value::String(self_string),
Some(serde_json::Value::String(other_string)),
_,
) => return self_string.partial_cmp(&other_string),
(
serde_json::Value::String(self_string),
_,
Some(OwnedLiteral::LString(other_string)),
) => return self_string.partial_cmp(&other_string),
// Otherwise, convert to numbers and compare them that way
(_, Some(json_other), _) => return compare_json_numbers(self, json_other),
(_, _, Some(literal_other)) => return compare_json_numbers(self, literal_other),
_ => panic!("Unimplemented comparison type."),
}
}
fn math_add<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
+ std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
+ std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_subtract<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
- std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
- std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_multiply<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
* std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
* std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_divide<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
/ std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
/ std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_modulus<'a>(&self, other: &dyn ContextElement) -> Option<IceResult<'a>> {
let other_json = other.to_any().downcast_ref::<Self>();
let other_literal = other.to_any().downcast_ref::<OwnedLiteral>();
match (self, other_json, other_literal) {
// If its neither of those types, then it is unimplemented
(_, None, None) => panic!("Math operation on unimplemented type"),
// Since this is specifically for the math helper, non-primitives are not supported
(serde_json::Value::Array(_), _, _)
| (serde_json::Value::Object(_), _, _)
| (_, Some(serde_json::Value::Array(_)), _)
| (_, Some(serde_json::Value::Object(_)), _) => None,
// Strings also are ignored because this is specifically a math function
(serde_json::Value::String(_), _, _)
| (_, Some(serde_json::Value::String(_)), _)
| (_, _, Some(OwnedLiteral::LString(_))) => None,
// Handle other serde_json::Value
(_, Some(other_json_value), _) => (std::convert::Into::<MathNumber>::into(self)
% std::convert::Into::<MathNumber>::into(other_json_value))
.map(IceResult::from_owned),
// Handle literals
(_, _, Some(other_literal)) => (std::convert::Into::<MathNumber>::into(self)
% std::convert::Into::<MathNumber>::into(other_literal))
.map(IceResult::from_owned),
}
}
fn math_abs<'a>(&self) -> Option<IceResult<'a>> {
std::convert::Into::<MathNumber>::into(self)
.math_abs()
.map(IceResult::from_owned)
}
fn math_floor<'a>(&self) -> Option<IceResult<'a>> {
std::convert::Into::<MathNumber>::into(self)
.math_floor()
.map(IceResult::from_owned)
}
fn math_ceil<'a>(&self) -> Option<IceResult<'a>> {
std::convert::Into::<MathNumber>::into(self)
.math_ceil()
.map(IceResult::from_owned)
}
}
impl From<&serde_json::Value> for ComparisonNumber {
/// Convert from a JSON value to a number for comparison based on
/// the logic described at
/// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Greater_than
fn from(original: &serde_json::Value) -> Self {
match original {
serde_json::Value::Null => ComparisonNumber::UnsignedInteger(0),
serde_json::Value::Bool(boolean) => {
if *boolean {
ComparisonNumber::UnsignedInteger(1)
} else {
ComparisonNumber::UnsignedInteger(0)
}
}
serde_json::Value::Number(num) => num.into(),
serde_json::Value::String(text) => text.into(),
serde_json::Value::Array(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
serde_json::Value::Object(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
}
}
}
impl From<&serde_json::Number> for ComparisonNumber {
fn from(original: &serde_json::Number) -> Self {
match original.as_u64() {
Some(num) => return ComparisonNumber::UnsignedInteger(num),
None => (),
};
match original.as_i64() {
Some(num) => return ComparisonNumber::SignedInteger(num),
None => (),
};
match original.as_f64() {
Some(num) => return ComparisonNumber::Decimal(num),
None => (),
};
ComparisonNumber::Failure
}
}
impl From<&serde_json::Value> for MathNumber {
fn from(original: &serde_json::Value) -> Self {
match original {
serde_json::Value::Null => MathNumber::Integer(0),
serde_json::Value::Bool(boolean) => {
if *boolean {
MathNumber::Integer(1)
} else {
MathNumber::Integer(0)
}
}
serde_json::Value::Number(num) => num.into(),
serde_json::Value::String(_) => {
panic!("Strings should not be cast to numbers for math")
}
serde_json::Value::Array(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
serde_json::Value::Object(_) => {
panic!("Only primitives should be cast to numbers for comparisons")
}
}
}
}
impl From<&serde_json::Number> for MathNumber {
fn from(original: &serde_json::Number) -> Self {
match original.as_u64() {
Some(num) => return MathNumber::Integer(num.try_into().unwrap()),
None => (),
};
match original.as_i64() {
Some(num) => return MathNumber::Integer(num.into()),
None => (),
};
match original.as_f64() {
Some(num) => return MathNumber::Decimal(num),
None => (),
};
MathNumber::Failure
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_nested_array_render() {
let x: serde_json::Value =
serde_json::from_str(r#"[3,5,[7,9]]"#).expect("Failed to parse json");
assert_eq!(
x.render(&Vec::new()),
Ok::<_, RenderError>("3,5,7,9".to_owned())
);
}
#[test]
fn test_html_escape() {
assert_eq!(html_escape("<>&\"'"), "&lt;&gt;&amp;&quot;&#39;".to_owned())
}
}

2
src/integrations/mod.rs Normal file
View File

@ -0,0 +1,2 @@
#[cfg(feature = "json-integration")]
pub mod json;

View File

@ -1,4 +1,5 @@
extern crate nom;
pub mod integrations;
pub mod parser;
pub mod renderer;

View File

@ -1,6 +1,7 @@
//! This module contains a rust implementation of LinkedIn Dust
mod parser;
mod take_until_parser_matches;
pub use parser::template;
pub use parser::Body;

View File

@ -1,8 +1,9 @@
use super::take_until_parser_matches::take_until_parser_matches;
use nom::branch::alt;
use nom::bytes::complete::escaped_transform;
use nom::bytes::complete::is_a;
use nom::bytes::complete::is_not;
use nom::bytes::complete::{tag, take_until, take_until_parser_matches};
use nom::bytes::complete::{tag, take_until};
use nom::character::complete::line_ending;
use nom::character::complete::multispace0;
use nom::character::complete::one_of;
@ -807,7 +808,10 @@ mod tests {
assert_eq!(super::special("{~rb}"), Ok(("", Special::RightCurlyBrace)));
assert_eq!(
super::special("{~zzz}"),
Err(Error(("zzz}", ErrorKind::Tag)))
Err(Error(nom::error::Error {
input: "zzz}",
code: ErrorKind::Tag
}))
);
}
@ -824,10 +828,10 @@ mod tests {
);
assert_eq!(
super::special("{! this is a comment without a close"),
Err(Error((
"{! this is a comment without a close",
ErrorKind::Tag
)))
Err(Error(nom::error::Error {
input: "{! this is a comment without a close",
code: ErrorKind::Tag
}))
);
}
@ -861,7 +865,10 @@ mod tests {
);
assert_eq!(
super::span("{~lb}"),
Err(Error(("{~lb}", ErrorKind::Verify)))
Err(Error(nom::error::Error {
input: "{~lb}",
code: ErrorKind::Verify
}))
);
assert_eq!(
super::body("this is \t \n\n \t \n \t multiline text\n {foo}"),
@ -911,7 +918,10 @@ mod tests {
fn test_section_mismatched_paths() {
assert_eq!(
super::dust_tag("{#foo.bar}{/baz}"),
Err(Error(("{#foo.bar}{/baz}", ErrorKind::Tag)))
Err(Error(nom::error::Error {
input: "{#foo.bar}{/baz}",
code: ErrorKind::Tag
}))
);
}
@ -1539,7 +1549,7 @@ mod tests {
{.}
{/names}"
),
Ok::<_, nom::Err<(&str, ErrorKind)>>((
Ok::<_, nom::Err<nom::error::Error<&str>>>((
"",
Template {
contents: Body {
@ -1613,7 +1623,7 @@ mod tests {
super::template(
r#"{#level3.level4}{>partialtwo v1="b" v2="b" v3="b" v4="b" v5="b" /}{/level3.level4}"#
),
Ok::<_, nom::Err<(&str, ErrorKind)>>((
Ok::<_, nom::Err<nom::error::Error<&str>>>((
"",
Template {
contents: Body {

View File

@ -0,0 +1,96 @@
use nom::{
error::ErrorKind, error::ParseError, IResult, InputIter, InputLength, InputTake, Parser,
};
/// Returns the shortest input slice till it matches the parser.
///
/// It doesn't consume the input to the parser. It will return `Err(Err::Error((_, ErrorKind::TakeUntilParserMatches)))`
/// if the pattern wasn't met
///
/// The performance of this parser depends HEAVILY on the inner parser
/// failing early. For each step on the input, this will run the inner
/// parser against the remaining input, so if the inner parser does
/// not fail fast then you will end up re-parsing the remaining input
/// repeatedly.
///
/// If you are looking to match until a string
/// (`take_until_parser_matches(tag("foo"))`) it would be faster to
/// use `take_until("foo")`.
///
/// # Simple Example
/// ```ignore
/// # #[macro_use] extern crate nom;
/// # use nom::{Err, error::ErrorKind, IResult};
/// use nom::bytes::complete::{take_until_parser_matches, tag};
///
/// fn until_eof(s: &str) -> IResult<&str, &str> {
/// take_until_parser_matches(tag("eof"))(s)
/// }
///
/// assert_eq!(until_eof("hello, worldeof"), Ok(("eof", "hello, world")));
/// assert_eq!(until_eof("hello, world"), Err(Err::Error(error_position!("hello, world", ErrorKind::TakeUntilParserMatches))));
/// assert_eq!(until_eof(""), Err(Err::Error(error_position!("", ErrorKind::TakeUntilParserMatches))));
/// ```
///
/// # Powerful Example
/// To show the power of this parser we will parse a line containing
/// a set of flags at the end surrounded by brackets. Example:
/// "Submit a PR [inprogress]"
/// ```ignore
/// # #[macro_use] extern crate nom;
/// # use nom::{Err, error::ErrorKind, IResult};
/// use nom::bytes::complete::{is_not, take_until_parser_matches, tag};
/// use nom::sequence::{delimited, tuple};
/// use nom::multi::separated_list1;
///
/// fn flag(i: &str) -> IResult<&str, &str> {
/// delimited(tag("["), is_not("]\r\n"), tag("]"))(i)
/// }
///
/// fn line_ending_with_flags(i: &str) -> IResult<&str, (&str, std::vec::Vec<&str>)> {
/// tuple((
/// take_until_parser_matches(flag),
/// separated_list1(tag(" "), flag),
/// ))(i)
/// }
///
/// assert_eq!(line_ending_with_flags("Parsing Seminar [important] [presentation]"), Ok(("", ("Parsing Seminar ", vec!["important", "presentation"]))));
/// ```
pub fn take_until_parser_matches<F, Input, O, Error>(
mut f: F,
) -> impl FnMut(Input) -> IResult<Input, Input, Error>
where
Input: InputTake + InputIter + InputLength + Clone,
F: Parser<Input, O, Error>,
Error: ParseError<Input>,
{
move |input: Input| {
let i = input.clone();
for (ind, _) in i.iter_indices() {
let (remaining, _taken) = i.take_split(ind);
match f.parse(remaining) {
Err(_) => (),
Ok(_) => {
let res: IResult<Input, Input, Error> = Ok(i.take_split(ind));
return res;
}
}
}
// Attempt to match one last time past the end of the input. This
// allows for 0-length combinators to be used (for example, an eof
// combinator).
let (remaining, _taken) = i.take_split(i.input_len());
match f.parse(remaining) {
Err(_) => (),
Ok(_) => {
let res: IResult<Input, Input, Error> = Ok(i.take_split(i.input_len()));
return res;
}
}
Err(nom::Err::Error(Error::from_error_kind(
i,
// Normally this would be `ErrorKind::TakeUntilParserMatches` but I cannot extend ErrorKind in this project.
ErrorKind::TakeUntil,
)))
}
}