Merge branch 'radio_target_properties'

I was only using Cow because regular links can use templates which would require owned data. Radio links do not share this feature.

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "organic"
-version = "0.1.8"
+version = "0.1.9"
 authors = ["Tom Alexander <tom@fizz.buzz>"]
 description = "An org-mode parser."
 edition = "2021"

Makefile

@@ -46,6 +46,13 @@ dockertest:
 > $(MAKE) -C docker/organic_test
 > docker run --init --rm -i -t --read-only -v "$$(readlink -f ./):/source:ro" --mount type=tmpfs,destination=/tmp --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source organic-test --no-default-features --features compare --no-fail-fast --lib --test test_loader -- --test-threads $(TESTJOBS)
 
+.PHONY: buildtest
+buildtest:
+> cargo build --no-default-features
+> cargo build --no-default-features --features compare
+> cargo build --no-default-features --features tracing
+> cargo build --no-default-features --features compare,tracing
+
 .PHONY: foreign_document_test
 foreign_document_test:
 > $(MAKE) -C docker/organic_test run_foreign_document_test

README.md

@@ -4,10 +4,31 @@ Organic is an emacs-less implementation of an [org-mode](https://orgmode.org/) p
 
 ## Project Status
 
-This project is a personal learning project to grow my experience in [rust](https://www.rust-lang.org/). It is under development and at this time I would not recommend anyone use this code. The goal is to turn this into a project others can use, at which point more information will appear in this README.
+This project is still under HEAVY development. While the version remains v0.1.x the API will be changing often. Once we hit v0.2.x we will start following semver.
+
+Currently, the parser is able to correctly identify the start/end bounds of all the org-mode objects and elements (except table.el tables, org-mode tables are supported) but many of the interior properties are not yet populated.
+
+### Project Goals
+- We aim to provide perfect parity with the emacs org-mode parser. In that regard, any document that parses differently between Emacs and Organic is considered a bug.
+- The parser should be fast. We're not doing anything special, but since this is written in Rust and natively compiled we should be able to beat the existing parsers.
+- The parser should have minimal dependencies. This should reduce effort w.r.t.: security audits, legal compliance, portability.
+- The parser should be usable everywhere. In the interest of getting org-mode used in as many places as possible, this parser should be usable by everyone everywhere. This means:
+  - It must have a permissive license for use in proprietary code bases.
+  - We will investigate compiling to WASM. This is an important goal of the project and will definitely happen, but only after the parser has a more stable API.
+  - We will investigate compiling to a C library for native linking to other code. This is more of a maybe-goal for the project.
+### Project Non-Goals
+- This project will not include an elisp engine since that would drastically increase the complexity of the code. Any features requiring an elisp engine will not be implemented (for example, Emacs supports embedded eval expressions in documents but this parser will never support that).
+- This project is exclusively an org-mode **parser**. This limits its scope to roughly the output of `(org-element-parse-buffer)`. It will not render org-mode documents in other formats like HTML or LaTeX.
+### Project Maybe-Goals
+- table.el support. Currently we support org-mode tables but org-mode also allows table.el tables. So far, their use in org-mode documents seems rather uncommon so this is a low-priority feature.
+- Document editing support. I do not anticipate any advanced editing features to make editing ergonomic, but it should be relatively easy to be able to parse an org-mode document and serialize it back into org-mode. This would enable cool features to be built on top of the library like auto-formatters. To accomplish this feature, We'd have to capture all of the various separators and whitespace that we are currently simply throwing away. This would add many additional fields to the parsed structs and it would add more noise to the parsers themselves, so I do not want to approach this feature until the parser is more complete since it would make modifications and refactoring more difficult.
+### Supported Versions
+This project targets the version of Emacs and Org-mode that are built into the [organic-test docker image](docker/organic_test/Dockerfile). This is newer than the version of Org-mode that shipped with Emacs 29.1. The parser itself does not depend on Emacs or Org-mode though, so this only matters for development purposes when running the automated tests that compare against upstream Org-mode.
 
 ## Using this library
-TODO: Add section on using Organic as a library (which is the intended use for this project).
+TODO: Add section on using Organic as a library (which is the intended use for this project). This will be added when we have a bit more API stability since currently the library is under heavy development.
+
+## Development
 
 ### The parse binary
 This program takes org-mode input either streamed in on stdin or as paths to files passed in as arguments. It then parses them using Organic and dumps the result to stdout. This program is intended solely as a development tool. Examples:

build.rs

@@ -19,8 +19,6 @@ fn main() {
     // Re-generate the tests if any org-mode files change
     println!("cargo:rerun-if-changed=org_mode_samples");
 
-    write_header(&mut test_file);
-
     let test_files = WalkDir::new("org_mode_samples")
         .into_iter()
         .filter(|e| match e {
@@ -54,28 +52,15 @@ fn write_test(test_file: &mut File, test: &walkdir::DirEntry) {
         .strip_suffix(".org")
         .expect("Should have .org extension")
         .replace("/", "_");
-    let test_name = format!("autogen_{}", test_name);
 
-    if let Some(_reason) = is_expect_fail(test_name.as_str()) {
-        write!(test_file, "#[ignore]\n").unwrap();
-    }
     write!(
         test_file,
         include_str!("./tests/test_template"),
         name = test_name,
-        path = test.path().display()
-    )
-    .unwrap();
-}
-
-#[cfg(feature = "compare")]
-fn write_header(test_file: &mut File) {
-    write!(
-        test_file,
-        r#"
-#[feature(exit_status_error)]
-
-"#
+        path = test.path().display(),
+        expect_fail = is_expect_fail(test_name.as_str())
+            .map(|_| "#[ignore]\n")
+            .unwrap_or("")
     )
     .unwrap();
 }
@@ -83,8 +68,8 @@ fn write_header(test_file: &mut File) {
 #[cfg(feature = "compare")]
 fn is_expect_fail(name: &str) -> Option<&str> {
     match name {
-        "autogen_greater_element_drawer_drawer_with_headline_inside" => Some("Apparently lines with :end: become their own paragraph. This odd behavior needs to be investigated more."),
-        "autogen_element_container_priority_footnote_definition_dynamic_block" => Some("Apparently broken begin lines become their own paragraph."),
+        "greater_element_drawer_drawer_with_headline_inside" => Some("Apparently lines with :end: become their own paragraph. This odd behavior needs to be investigated more."),
+        "element_container_priority_footnote_definition_dynamic_block" => Some("Apparently broken begin lines become their own paragraph."),
         _ => None,
     }
 }

docker/organic_test/Dockerfile

@@ -88,7 +88,7 @@ ARG DOOMEMACS_PATH=/foreign_documents/doomemacs
 ARG DOOMEMACS_REPO=https://github.com/doomemacs/doomemacs.git
 RUN mkdir -p $DOOMEMACS_PATH && git -C $DOOMEMACS_PATH init --initial-branch=main && git -C $DOOMEMACS_PATH remote add origin $DOOMEMACS_REPO && git -C $DOOMEMACS_PATH fetch origin $DOOMEMACS_VERSION && git -C $DOOMEMACS_PATH checkout FETCH_HEAD
 
-ARG WORG_VERSION=0c8d5679b536af450b61812246a3e02b8103f4b8
+ARG WORG_VERSION=ba6cda890f200d428a5d68e819eef15b5306055f
 ARG WORG_PATH=/foreign_documents/worg
 ARG WORG_REPO=https://git.sr.ht/~bzg/worg
 RUN mkdir -p $WORG_PATH && git -C $WORG_PATH init --initial-branch=main && git -C $WORG_PATH remote add origin $WORG_REPO && git -C $WORG_PATH fetch origin $WORG_VERSION && git -C $WORG_PATH checkout FETCH_HEAD

notes/test_names.org

@@ -0,0 +1,7 @@
+* Autogen tests
+The autogen tests are the tests automatically generated to compare the output of Organic vs the upstream Emacs Org-mode parser using the sample documents in the =org_mode_samples= folder. They will have a prefix based on the settings for each test.
+
+- default :: The test is run with the default settings (The upstream Emacs Org-mode determines the default settings)
+- la :: Short for "list alphabetic". Enables alphabetic plain lists.
+- t# :: Sets the tab-width to # (as in t4 sets the tab-width to 4).
+- odd :: Sets the org-odd-levels-only setting to true (meaning "odd" as opposed to "oddeven").

org_mode_samples/affiliated_keyword/all_name.org

@@ -0,0 +1,17 @@
+#+name: foo
+: bar
+
+#+source: foo
+: bar
+
+#+tblname: foo
+: bar
+
+#+resname: foo
+: bar
+
+#+srcname: foo
+: bar
+
+#+label: foo
+: bar

org_mode_samples/affiliated_keyword/case_insensitive.org

@@ -0,0 +1,8 @@
+#+NAME: foo
+bar
+
+#+NaMe: baz
+cat
+
+#+name: lorem
+ipsum

org_mode_samples/affiliated_keyword/multiple_name.org

@@ -0,0 +1,6 @@
+#+name: foo
+#+source: bar
+#+name: baz
+#+tblname: lorem
+#+label: ipsum
+: dolar

org_mode_samples/affiliated_keyword/tblname_on_non_table.org

@@ -0,0 +1,2 @@
+#+tblname: foo
+: bar

org_mode_samples/document/category.org

@@ -0,0 +1 @@
+#+CATEGORY: theory

org_mode_samples/document/category_multiple.org

@@ -0,0 +1,5 @@
+#+CATEGORY: foo
+#+CATEGORY: bar
+#+begin_src text
+#+CATEGORY: baz
+#+end_src

org_mode_samples/greater_element/drawer/name.org

@@ -0,0 +1,10 @@
+* Headline
+before
+#+NAME: foo
+:candle:
+inside
+
+the drawer
+:end:
+
+after

org_mode_samples/greater_element/dynamic_block/name.org

@@ -0,0 +1,7 @@
+#+NAME: foo
+#+BEGIN: clocktable :scope file :maxlevel 2
+#+CAPTION: Clock summary at [2023-08-25 Fri 05:34]
+| Headline     | Time   |
+|--------------+--------|
+| *Total time* | *0:00* |
+#+END:

org_mode_samples/greater_element/greater_block/center/empty_with_name_inside.org

@@ -0,0 +1,8 @@
+#+begin_center
+
+#+end_center
+
+#+begin_center
+#+NAME: foo
+
+#+end_center

org_mode_samples/greater_element/greater_block/center/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_center
+
+#+end_center

org_mode_samples/greater_element/greater_block/quote/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_quote
+
+#+end_quote

org_mode_samples/greater_element/greater_block/special/name.org

@@ -0,0 +1,6 @@
+#+NAME: foo
+#+begin_defun
+foo
+
+{{{bar(baz)}}}
+#+end_defun

org_mode_samples/greater_element/greater_block/with_params.org

@@ -0,0 +1,3 @@
+#+begin_defun foo bar baz   
+lorem
+#+end_defun

org_mode_samples/greater_element/plain_list/alphabetic_counter_set.org

@@ -0,0 +1,22 @@
+# An ordered list starting at 3
+1. [@3] foo
+
+
+# An ordered list starting at 11
+1. [@D] bar
+
+
+# An ordered list starting at 1 with the contents of "[@kk] baz"
+1. [@kk] baz
+
+
+# A paragraph when org-list-allow-alphabetical is nil
+m. lorem
+
+
+# A paragraph when org-list-allow-alphabetical is nil
+m. [@k] ipsum
+
+
+# An unordered list with :counter set to 3
+- [@3] dolar

org_mode_samples/greater_element/plain_list/alphabetic_list_larger_than_26.org

@@ -0,0 +1,30 @@
+# Alphabetic lists larger than 26 elements should become numbered. From M-x describe-variable org-list-allow-alphabetical:
+#
+# > Lists with more than 26 items will fallback to standard numbering.
+a. 1
+a. 2
+a. 3
+a. 4
+a. 5
+a. 6
+a. 7
+a. 8
+a. 9
+a. 10
+a. 11
+a. 12
+a. 13
+a. 14
+a. 15
+a. 16
+a. 17
+a. 18
+a. 19
+a. 20
+a. 21
+a. 22
+a. 23
+a. 24
+a. 25
+a. 26
+a. 27

org_mode_samples/greater_element/plain_list/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+1. bar

org_mode_samples/greater_element/plain_list/tab_width_indentation_level.org

@@ -0,0 +1,5 @@
+# "lorem" is prefixed by a tab instead of spaces, so the editor's tab-width value determines whether lorem is a sibling of baz (tab-width 8), a sibling of bar (tab-width < 8), or a child of baz (tab-width > 8).
+1. foo
+ 1. bar
+        1. baz
+	1. lorem

org_mode_samples/greater_element/table/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+| foo | bar |

org_mode_samples/lesser_element/babel_call/empty.org

@@ -0,0 +1,3 @@
+#+call:
+
+#+call:    

org_mode_samples/lesser_element/babel_call/header_arguments.org

@@ -0,0 +1,3 @@
+#+call: foo[inside](bar="baz")[outside]
+
+#+call: foo[](bar="baz")[]

org_mode_samples/lesser_element/babel_call/name.org

@@ -1 +1,2 @@
+#+NAME: foo
 #+call: foo(bar="baz")

org_mode_samples/lesser_element/babel_call/nested_brackets.org

@@ -0,0 +1,7 @@
+#+call: foo[inside](bar="baz")[outside]
+
+#+call: foo[[inside]](bar="baz")[outside]
+
+#+call: foo[inside]((bar="baz"))[outside]
+
+#+call: foo[inside](bar="baz")[[outside]]

org_mode_samples/lesser_element/babel_call/no_closing_parenthesis.org

@@ -0,0 +1 @@
+#+call: foo(bar="baz"

org_mode_samples/lesser_element/babel_call/simple.org

@@ -0,0 +1,7 @@
+#+call: foo(bar="baz")
+
+#+call: lorem ipsum
+
+#+call: dolar cat(dog)
+
+#+call: (bat)

org_mode_samples/lesser_element/babel_call/spaces.org

@@ -0,0 +1,3 @@
+#+call: foo [inside] (bar="baz") [outside] 
+
+#+call: foo (bar="baz") [outside] 

org_mode_samples/lesser_element/clock/date_range_without_duration.org

@@ -0,0 +1 @@
+CLOCK: [2023-04-21 Fri 19:32]--[2023-04-21 Fri 19:35]

org_mode_samples/lesser_element/clock/name.org

@@ -0,0 +1,3 @@
+CLOCK: [2023-04-21 Fri 19:32]--[2023-04-21 Fri 19:35] =>  0:03
+#+NAME: foo
+CLOCK: [2023-04-21 Fri 19:43]

org_mode_samples/lesser_element/clock/non_range_with_duration.org

@@ -0,0 +1 @@
+CLOCK: [2023-04-21 Fri 19:43] =>  0:03

org_mode_samples/lesser_element/clock/time_range.org

@@ -0,0 +1,2 @@
+CLOCK: [1970-01-01 Thu 8:15-13:15otherrest +1w -1d] =>  0:03
+CLOCK: [1970-01-01 Thu 8:15-13:15otherrest +1w -1d]

org_mode_samples/lesser_element/comment/indented.org

@@ -1,4 +1,5 @@
 # Comment
+#   
   # indented line
 #        At the top of the file
 

org_mode_samples/lesser_element/comment/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+# Comments cannot have affiliated keywords.

org_mode_samples/lesser_element/diary_sexp/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+%%(foo)

org_mode_samples/lesser_element/fixed_width_area/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+: bar

org_mode_samples/lesser_element/horizontal_rule/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+-----

org_mode_samples/lesser_element/keyword/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+#+FOO: BAR

org_mode_samples/lesser_element/latex_environment/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+\begin{foo}
+bar
+\end{foo}

org_mode_samples/lesser_element/lesser_block/comment/escaped_lines.org

@@ -0,0 +1,10 @@
+#+begin_comment
+,* foo
+,,,** bar
+   ,*** baz
+    lorem
+    , ipsum
+,#+begin_src dolar
+
+,#+end_src
+#+end_comment

org_mode_samples/lesser_element/lesser_block/comment/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_comment text
+bar
+#+end_comment

org_mode_samples/lesser_element/lesser_block/comment/with_comma.org

@@ -0,0 +1,4 @@
+#+begin_comment
+This is a comment
+,* with an escaped line.
+#+end_comment

org_mode_samples/lesser_element/lesser_block/comment/with_switches.org

@@ -0,0 +1,3 @@
+#+begin_comment -n 20
+foo
+#+end_comment

org_mode_samples/lesser_element/lesser_block/example/escaped_lines.org

@@ -0,0 +1,10 @@
+#+begin_example
+,* foo
+,,,** bar
+   ,*** baz
+    lorem
+    , ipsum
+,#+begin_src dolar
+
+,#+end_src
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/exports_flag.org

@@ -0,0 +1,7 @@
+#+begin_example python :exports results
+print("foo")
+#+end_example
+
+#+begin_example python -n :exports results
+print("foo")
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/language_and_switches.org

@@ -0,0 +1,3 @@
+#+begin_example elisp -n 5
+foo
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/line_number_negative.org

@@ -0,0 +1,7 @@
+#+begin_example -n -10
+foo
+#+end_example
+
+#+begin_example +n -15
+bar
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/line_number_zero.org

@@ -0,0 +1,7 @@
+#+begin_example -n 0
+foo
+#+end_example
+
+#+begin_example +n 0
+bar
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/line_numbers.org

@@ -0,0 +1,8 @@
+#+begin_example -n 5
+foo
+#+end_example
+
+# Line numbering starts at 15 for the example below since it uses +n.
+#+begin_example +n 10
+bar
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_example text
+bar
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/only_k.org

@@ -0,0 +1,3 @@
+#+begin_example foo -k
+bar
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/parameters_without_colon.org

@@ -0,0 +1,3 @@
+#+begin_example foo -n bar -k baz
+
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/preserve_indent.org

@@ -0,0 +1,15 @@
+#+begin_example text -i
+     foo
+#+end_example
+
+#+begin_example text    -n -i
+     foo
+#+end_example
+
+#+begin_example text
+     foo
+#+end_example
+
+#+begin_example text    -n -r -k
+     foo
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/reference.org

@@ -0,0 +1,6 @@
+#+begin_example text
+  foo
+  bar (ref:here)
+  baz
+#+end_example
+Link to the reference: [[(here)]]

org_mode_samples/lesser_element/lesser_block/example/space_for_parameters.org

@@ -0,0 +1,2 @@
+#+begin_example 
+#+end_example

org_mode_samples/lesser_element/lesser_block/example/switches_with_label.org

@@ -0,0 +1,15 @@
+#+BEGIN_EXAMPLE elisp -n -r -l "((%s))"
+   foo
+#+END_EXAMPLE
+
+#+BEGIN_EXAMPLE elisp -k -n -r -l "((%s))"
+   foo
+#+END_EXAMPLE
+
+#+BEGIN_EXAMPLE elisp -k 8 -n -r -l "((%s))"
+   foo
+#+END_EXAMPLE
+
+#+BEGIN_EXAMPLE elisp -n -r -k -l "((%s))"
+   foo
+#+END_EXAMPLE

org_mode_samples/lesser_element/lesser_block/example/switches_with_trailing_whitespace.org

@@ -0,0 +1,3 @@
+#+begin_example +n 10     
+foo
+#+end_example

org_mode_samples/lesser_element/lesser_block/export/escaped_lines.org

@@ -0,0 +1,10 @@
+#+begin_export html
+,* foo
+,,,** bar
+   ,*** baz
+    lorem
+    , ipsum
+,#+begin_src dolar
+
+,#+end_src
+#+end_export

org_mode_samples/lesser_element/lesser_block/export/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_export text
+bar
+#+end_export

org_mode_samples/lesser_element/lesser_block/export/one_word.org

@@ -0,0 +1,3 @@
+#+begin_export latex
+This would be LaTeX code.
+#+end_export

org_mode_samples/lesser_element/lesser_block/nested.org

@@ -0,0 +1,6 @@
+# Verse blocks are the only lesser blocks that contain objects
+#+begin_verse
+#+begin_comment
+This is a comment.
+#+end_comment
+#+end_verse

org_mode_samples/lesser_element/lesser_block/src/exports_flag.org

@@ -0,0 +1,7 @@
+#+begin_src python :exports results
+print("foo")
+#+end_src
+
+#+begin_src python -n :exports results
+print("foo")
+#+end_src

org_mode_samples/lesser_element/lesser_block/src/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_src text
+bar
+#+end_src

org_mode_samples/lesser_element/lesser_block/src/parameters_with_trailing_space.org

@@ -0,0 +1,7 @@
+#+begin_src python :exports results 
+print("foo")
+#+end_src
+
+#+begin_src python -n :exports results 
+print("foo")
+#+end_src

org_mode_samples/lesser_element/lesser_block/src/parameters_without_colon.org

@@ -0,0 +1,3 @@
+#+begin_src foo -n bar -k baz
+
+#+end_src

org_mode_samples/lesser_element/lesser_block/src/preserve_indent.org

@@ -0,0 +1,15 @@
+#+begin_src text -i
+     foo
+#+end_src
+
+#+begin_src text    -n -i
+     foo
+#+end_src
+
+#+begin_src text
+     foo
+#+end_src
+
+#+begin_src text    -n -r -k
+     foo
+#+end_src

org_mode_samples/lesser_element/lesser_block/src/reference.org

@@ -0,0 +1,6 @@
+#+begin_src text
+  foo
+  bar (ref:here)
+  baz
+#+end_src
+Link to the reference: [[(here)]]

org_mode_samples/lesser_element/lesser_block/verse/name.org

@@ -0,0 +1,4 @@
+#+NAME: foo
+#+begin_verse text
+bar
+#+end_verse

org_mode_samples/lesser_element/paragraph/name.org

@@ -0,0 +1,2 @@
+#+NAME: foo
+bar

org_mode_samples/object/regular_link/code_ref_link.org

@@ -0,0 +1,13 @@
+[[(foo)]]
+
+[[((bar))]]
+
+[[((baz)]]
+
+[[(lo
+rem)]]
+
+# These become fuzzy
+[[(foo) ]]
+[[ (foo)]]
+[[(foo)::3]]

org_mode_samples/object/regular_link/custom_id_link.org

@@ -0,0 +1,6 @@
+[[#foo]]
+
+[[#fo
+o]]
+
+[[#foo::3]]

org_mode_samples/object/regular_link/file_link.org

@@ -0,0 +1,21 @@
+[[./simple.org]]
+[[../simple.org]]
+[[/simple.org]]
+[[file:simple.org]]
+[[file:sim  ple.org]]
+
+[[file:simp
+le.org]]
+
+[[file:simple.org::3]]
+[[file:simple.org::foo]]
+[[file:simple.org::#foo]]
+[[file:simple.org::foo  bar]]
+[[file:simple.org::foo
+bar]]
+[[file:simple.org::foo  
+bar]]
+[[file:simple.org::foo  
+  bar]]
+[[file:simple.org::foo::bar]]
+[[file:simple.org::/foo/]]

org_mode_samples/object/regular_link/fuzzy_link.org

@@ -0,0 +1,6 @@
+[[elisp.org]]
+
+[[eli
+sp.org]]
+
+[[elisp.org::3]]

org_mode_samples/object/regular_link/id_link.org

@@ -0,0 +1,6 @@
+[[id:83986bdf-987c-465d-8851-44cb4c02a86c]]
+
+[[id:83986bdf-987c-465d
+-8851-44cb4c02a86c]]
+
+[[id:83986bdf-987c-465d-8851-44cb4c02a86c::foo]]

org_mode_samples/object/regular_link/protocol_link.org

@@ -0,0 +1,6 @@
+[[shell:foo]]
+
+[[shell:fo
+o]]
+
+[[shell:foo::3]]

org_mode_samples/object/regular_link/template.org

@@ -0,0 +1,7 @@
+#+LINK:       foo https://foo.bar/baz#%s
+[[foo::lorem]]
+[[foo::ipsum][dolar]]
+
+[[cat::bat]]
+#+LINK:       cat dog%s
+[[cat:bat]]

org_mode_samples/object/timestamp/all_repeater_warning_delay_mark.org

@@ -0,0 +1,7 @@
+# All the marks for repeater and warning delay
+[1970-01-01 Thu 8:15-13:15foo +1h -2h]
+[1970-01-01 Thu 8:15-13:15foo ++1d -2d]
+[1970-01-01 Thu 8:15-13:15foo .+1w -2w]
+[1970-01-01 Thu 8:15-13:15foo +1m --2m]
+[1970-01-01 Thu 8:15-13:15foo ++1y --2y]
+[1970-01-01 Thu 8:15-13:15foo .+1d --2h]

org_mode_samples/object/timestamp/simple.org

@@ -2,13 +2,17 @@
 <%%(foo bar baz)>
 # active
 <1970-01-01 Thu 8:15rest +1w -1d>
+# Any value for "REST" in the first timestamp makes this a regular timestamp rather than a time range.
+<1970-01-01 Thu 8:15rest-13:15otherrest +1w -1d>
 # inactive
 [1970-01-01 Thu 8:15rest +1w -1d]
+# Any value for "REST" in the first timestamp makes this a regular timestamp rather than a time range.
+[1970-01-01 Thu 8:15rest-13:15otherrest +1w -1d]
 # active date range
 <1970-01-01 Thu 8:15rest +1w -1d>--<1970-01-01 Thu 8:15rest +1w -1d>
 # active time range
-<1970-01-01 Thu 8:15rest-13:15otherrest +1w -1d>
+<1970-01-01 Thu 8:15-13:15otherrest +1w -1d>
 # inactive date range
 [1970-01-01 Thu 8:15rest +1w -1d]--[1970-01-01 Thu 8:15rest +1w -1d]
 # inactive time range
-[1970-01-01 Thu 8:15rest-13:15otherrest +1w -1d]
+[1970-01-01 Thu 8:15-13:15otherrest +1w -1d]

org_mode_samples/object/timestamp/timeless_rest.org

@@ -0,0 +1,2 @@
+# This should be a malformed timestamp according to the current org-mode documentation but it is accepted anyway (with no repeater).
+<1970-01-01 Thu ++y>

org_mode_samples/sections_and_headings/footnote_section/as_target.org

@@ -0,0 +1,2 @@
+<<<Footnotes>>> and stuff
+* Footnotes

org_mode_samples/sections_and_headings/footnote_section/different_case.org

@@ -0,0 +1,3 @@
+* FOOTNOTES
+* Footnotes
+* footnotes

org_mode_samples/sections_and_headings/footnote_section/multiple.org

@@ -0,0 +1,2 @@
+* Footnotes
+* Footnotes

org_mode_samples/sections_and_headings/footnote_section/simple.org

@@ -0,0 +1,3 @@
+* Foo
+* Footnotes   :foo:bar:
+* Footnotes and stuff

scripts/callgrind.bash

@@ -4,10 +4,23 @@ set -euo pipefail
 IFS=$'\n\t'
 DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 
+: ${PROFILE:="perf"}
 
+function main {
+    local additional_flags=()
+    if [ "$PROFILE" = "dev" ] || [ "$PROFILE" = "debug" ]; then
+        PROFILE="debug"
+    else
+        additional_flags+=(--profile "$PROFILE")
+        # We have to disable avx512 because valgrind does not yet support it.
+        export RUSTFLAGS="-C target-feature=-avx512"
+    fi
 
-(cd "$DIR/../" && RUSTFLAGS="-C opt-level=0" cargo build --no-default-features)
-valgrind --tool=callgrind --callgrind-out-file="$DIR/../callgrind.out" "$DIR/../target/debug/parse" "${@}"
+    (cd "$DIR/../" && RUSTFLAGS="-C target-cpu=x86-64-v3" cargo build --no-default-features "${additional_flags[@]}")
+    valgrind --tool=callgrind --callgrind-out-file="$DIR/../callgrind.out" "$DIR/../target/${PROFILE}/parse" "${@}"
 
     echo "You probably want to run:"
     echo "callgrind_annotate --auto=yes '$DIR/../callgrind.out'"
+}
+
+main "${@}"

src/compare/compare.rs

@@ -1,26 +1,37 @@
 use std::path::Path;
 
 use crate::compare::diff::compare_document;
+use crate::compare::diff::DiffResult;
 use crate::compare::parse::emacs_parse_anonymous_org_document;
 use crate::compare::parse::emacs_parse_file_org_document;
 use crate::compare::parse::get_emacs_version;
 use crate::compare::parse::get_org_mode_version;
 use crate::compare::sexp::sexp;
-use crate::parser::parse;
+use crate::context::GlobalSettings;
+use crate::context::LocalFileAccessInterface;
+use crate::parser::parse_file_with_settings;
 use crate::parser::parse_with_settings;
-use crate::GlobalSettings;
-use crate::LocalFileAccessInterface;
 
 pub fn run_anonymous_compare<P: AsRef<str>>(
     org_contents: P,
+) -> Result<(), Box<dyn std::error::Error>> {
+    run_anonymous_compare_with_settings(org_contents, &GlobalSettings::default())
+}
+
+pub fn run_compare_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn std::error::Error>> {
+    run_compare_on_file_with_settings(org_path, &GlobalSettings::default())
+}
+
+pub fn run_anonymous_compare_with_settings<P: AsRef<str>>(
+    org_contents: P,
+    global_settings: &GlobalSettings,
 ) -> Result<(), Box<dyn std::error::Error>> {
     // TODO: This is a work-around to pretend that dos line endings do not exist. It would be better to handle the difference in line endings.
     let org_contents = org_contents.as_ref().replace("\r\n", "\n");
     let org_contents = org_contents.as_str();
-    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
-    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
-    let rust_parsed = parse(org_contents)?;
-    let org_sexp = emacs_parse_anonymous_org_document(org_contents)?;
+    print_versions()?;
+    let rust_parsed = parse_with_settings(org_contents, global_settings)?;
+    let org_sexp = emacs_parse_anonymous_org_document(org_contents, global_settings)?;
     let (_remaining, parsed_sexp) = sexp(org_sexp.as_str()).map_err(|e| e.to_string())?;
 
     println!("{}\n\n\n", org_contents);
@@ -33,15 +44,23 @@ pub fn run_anonymous_compare<P: AsRef<str>>(
 
     if diff_result.is_bad() {
         Err("Diff results do not match.")?;
+    } else {
+        println!(
+            "{color}Entire document passes.{reset}",
+            color = DiffResult::foreground_color(0, 255, 0),
+            reset = DiffResult::reset_color(),
+        );
     }
 
     Ok(())
 }
 
-pub fn run_compare_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn std::error::Error>> {
+pub fn run_compare_on_file_with_settings<P: AsRef<Path>>(
+    org_path: P,
+    global_settings: &GlobalSettings,
+) -> Result<(), Box<dyn std::error::Error>> {
     let org_path = org_path.as_ref();
-    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
-    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
+    print_versions()?;
     let parent_directory = org_path
         .parent()
         .ok_or("Should be contained inside a directory.")?;
@@ -53,12 +72,12 @@ pub fn run_compare_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn st
         working_directory: Some(parent_directory.to_path_buf()),
     };
     let global_settings = {
-        let mut global_settings = GlobalSettings::default();
+        let mut global_settings = global_settings.clone();
         global_settings.file_access = &file_access_interface;
         global_settings
     };
-    let rust_parsed = parse_with_settings(org_contents, &global_settings)?;
-    let org_sexp = emacs_parse_file_org_document(org_path)?;
+    let rust_parsed = parse_file_with_settings(org_contents, &global_settings, Some(org_path))?;
+    let org_sexp = emacs_parse_file_org_document(org_path, &global_settings)?;
     let (_remaining, parsed_sexp) = sexp(org_sexp.as_str()).map_err(|e| e.to_string())?;
 
     println!("{}\n\n\n", org_contents);
@@ -71,7 +90,19 @@ pub fn run_compare_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn st
 
     if diff_result.is_bad() {
         Err("Diff results do not match.")?;
+    } else {
+        println!(
+            "{color}Entire document passes.{reset}",
+            color = DiffResult::foreground_color(0, 255, 0),
+            reset = DiffResult::reset_color(),
+        );
     }
 
     Ok(())
 }
+
+fn print_versions() -> Result<(), Box<dyn std::error::Error>> {
+    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
+    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
+    Ok(())
+}

src/compare/compare_field.rs

@@ -0,0 +1,104 @@
+use std::fmt::Debug;
+
+use super::diff::DiffStatus;
+use super::sexp::Token;
+use super::util::get_property;
+use super::util::get_property_quoted_string;
+use super::util::get_property_unquoted_atom;
+
+#[derive(Debug)]
+pub(crate) enum EmacsField<'s> {
+    Required(&'s str),
+    #[allow(dead_code)]
+    Optional(&'s str),
+}
+
+/// Do no comparison.
+///
+/// This is for when you want to acknowledge that a field exists in the emacs token, but you do not have any validation for it when using the compare_properties!() macro. Ideally, this should be kept to a minimum since this represents untested values.
+#[allow(dead_code)]
+pub(crate) fn compare_noop<'b, 's, 'x, R, RG>(
+    _emacs: &'b Token<'s>,
+    _rust_node: R,
+    _emacs_field: &'x str,
+    _rust_value_getter: RG,
+) -> Result<Option<(DiffStatus, Option<String>)>, Box<dyn std::error::Error>> {
+    Ok(None)
+}
+
+/// Do no comparison.
+///
+/// This is for when you want to acknowledge that a field exists in the emacs token, but you do not have any validation for it when using the compare_properties!() macro. Ideally, this should be kept to a minimum since this represents untested values.
+#[allow(dead_code)]
+pub(crate) fn compare_identity() -> () {
+    ()
+}
+
+/// Assert that the emacs value is always nil or absent.
+///
+/// This is usually used for fields which, in my testing, are always nil. Using this compare function instead of simply doing a compare_noop will enable us to be alerted when we finally come across an org-mode document that has a value other than nil for the property.
+pub(crate) fn compare_property_always_nil<'b, 's, 'x, R, RG>(
+    emacs: &'b Token<'s>,
+    _rust_node: R,
+    emacs_field: &'x str,
+    _rust_value_getter: RG,
+) -> Result<Option<(DiffStatus, Option<String>)>, Box<dyn std::error::Error>> {
+    let value = get_property(emacs, emacs_field)?;
+    if value.is_some() {
+        let this_status = DiffStatus::Bad;
+        let message = Some(format!(
+            "{} was expected to always be nil: {:?}",
+            emacs_field, value
+        ));
+        Ok(Some((this_status, message)))
+    } else {
+        Ok(None)
+    }
+}
+
+pub(crate) fn compare_property_quoted_string<
+    'b,
+    's,
+    'x,
+    R,
+    RV: AsRef<str> + std::fmt::Debug,
+    RG: Fn(R) -> Option<RV>,
+>(
+    emacs: &'b Token<'s>,
+    rust_node: R,
+    emacs_field: &'x str,
+    rust_value_getter: RG,
+) -> Result<Option<(DiffStatus, Option<String>)>, Box<dyn std::error::Error>> {
+    let value = get_property_quoted_string(emacs, emacs_field)?;
+    let rust_value = rust_value_getter(rust_node);
+    if rust_value.as_ref().map(|s| s.as_ref()) != value.as_ref().map(String::as_str) {
+        let this_status = DiffStatus::Bad;
+        let message = Some(format!(
+            "{} mismatch (emacs != rust) {:?} != {:?}",
+            emacs_field, value, rust_value
+        ));
+        Ok(Some((this_status, message)))
+    } else {
+        Ok(None)
+    }
+}
+
+pub(crate) fn compare_property_unquoted_atom<'b, 's, 'x, R, RG: Fn(R) -> Option<&'s str>>(
+    emacs: &'b Token<'s>,
+    rust_node: R,
+    emacs_field: &'x str,
+    rust_value_getter: RG,
+) -> Result<Option<(DiffStatus, Option<String>)>, Box<dyn std::error::Error>> {
+    let value = get_property_unquoted_atom(emacs, emacs_field)?;
+    let rust_value = rust_value_getter(rust_node);
+    if rust_value != value {
+        let this_status = DiffStatus::Bad;
+        let message = Some(format!(
+            "{} mismatch (emacs != rust) {:?} != {:?}",
+            emacs_field, value, rust_value
+        ));
+        Ok(Some((this_status, message)))
+    } else {
+        Ok(None)
+    }
+}

src/compare/elisp_fact.rs

@@ -0,0 +1,552 @@
+use std::borrow::Cow;
+
+use crate::types::AngleLink;
+use crate::types::AstNode;
+use crate::types::BabelCall;
+use crate::types::Bold;
+use crate::types::CenterBlock;
+use crate::types::Citation;
+use crate::types::CitationReference;
+use crate::types::Clock;
+use crate::types::Code;
+use crate::types::Comment;
+use crate::types::CommentBlock;
+use crate::types::DiarySexp;
+use crate::types::Document;
+use crate::types::Drawer;
+use crate::types::DynamicBlock;
+use crate::types::Element;
+use crate::types::Entity;
+use crate::types::ExampleBlock;
+use crate::types::ExportBlock;
+use crate::types::ExportSnippet;
+use crate::types::FixedWidthArea;
+use crate::types::FootnoteDefinition;
+use crate::types::FootnoteReference;
+use crate::types::Heading;
+use crate::types::HorizontalRule;
+use crate::types::InlineBabelCall;
+use crate::types::InlineSourceBlock;
+use crate::types::Italic;
+use crate::types::Keyword;
+use crate::types::LatexEnvironment;
+use crate::types::LatexFragment;
+use crate::types::LineBreak;
+use crate::types::NodeProperty;
+use crate::types::Object;
+use crate::types::OrgMacro;
+use crate::types::Paragraph;
+use crate::types::PlainLink;
+use crate::types::PlainList;
+use crate::types::PlainListItem;
+use crate::types::PlainText;
+use crate::types::Planning;
+use crate::types::PropertyDrawer;
+use crate::types::QuoteBlock;
+use crate::types::RadioLink;
+use crate::types::RadioTarget;
+use crate::types::RegularLink;
+use crate::types::Section;
+use crate::types::SpecialBlock;
+use crate::types::SrcBlock;
+use crate::types::StatisticsCookie;
+use crate::types::StrikeThrough;
+use crate::types::Subscript;
+use crate::types::Superscript;
+use crate::types::Table;
+use crate::types::TableCell;
+use crate::types::TableRow;
+use crate::types::Target;
+use crate::types::Timestamp;
+use crate::types::Underline;
+use crate::types::Verbatim;
+use crate::types::VerseBlock;
+
+pub(crate) trait ElispFact<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str>;
+}
+
+pub(crate) trait GetElispFact<'s> {
+    fn get_elisp_fact<'b>(&'b self) -> &'b dyn ElispFact<'s>;
+}
+
+impl<'s, I: ElispFact<'s>> GetElispFact<'s> for I {
+    fn get_elisp_fact<'b>(&'b self) -> &'b dyn ElispFact<'s> {
+        self
+    }
+}
+
+impl<'r, 's> GetElispFact<'s> for AstNode<'r, 's> {
+    fn get_elisp_fact<'b>(&'b self) -> &'b dyn ElispFact<'s> {
+        match self {
+            AstNode::Document(inner) => *inner,
+            AstNode::Heading(inner) => *inner,
+            AstNode::Section(inner) => *inner,
+            AstNode::Paragraph(inner) => *inner,
+            AstNode::PlainList(inner) => *inner,
+            AstNode::PlainListItem(inner) => *inner,
+            AstNode::CenterBlock(inner) => *inner,
+            AstNode::QuoteBlock(inner) => *inner,
+            AstNode::SpecialBlock(inner) => *inner,
+            AstNode::DynamicBlock(inner) => *inner,
+            AstNode::FootnoteDefinition(inner) => *inner,
+            AstNode::Comment(inner) => *inner,
+            AstNode::Drawer(inner) => *inner,
+            AstNode::PropertyDrawer(inner) => *inner,
+            AstNode::NodeProperty(inner) => *inner,
+            AstNode::Table(inner) => *inner,
+            AstNode::TableRow(inner) => *inner,
+            AstNode::VerseBlock(inner) => *inner,
+            AstNode::CommentBlock(inner) => *inner,
+            AstNode::ExampleBlock(inner) => *inner,
+            AstNode::ExportBlock(inner) => *inner,
+            AstNode::SrcBlock(inner) => *inner,
+            AstNode::Clock(inner) => *inner,
+            AstNode::DiarySexp(inner) => *inner,
+            AstNode::Planning(inner) => *inner,
+            AstNode::FixedWidthArea(inner) => *inner,
+            AstNode::HorizontalRule(inner) => *inner,
+            AstNode::Keyword(inner) => *inner,
+            AstNode::BabelCall(inner) => *inner,
+            AstNode::LatexEnvironment(inner) => *inner,
+            AstNode::Bold(inner) => *inner,
+            AstNode::Italic(inner) => *inner,
+            AstNode::Underline(inner) => *inner,
+            AstNode::StrikeThrough(inner) => *inner,
+            AstNode::Code(inner) => *inner,
+            AstNode::Verbatim(inner) => *inner,
+            AstNode::PlainText(inner) => *inner,
+            AstNode::RegularLink(inner) => *inner,
+            AstNode::RadioLink(inner) => *inner,
+            AstNode::RadioTarget(inner) => *inner,
+            AstNode::PlainLink(inner) => *inner,
+            AstNode::AngleLink(inner) => *inner,
+            AstNode::OrgMacro(inner) => *inner,
+            AstNode::Entity(inner) => *inner,
+            AstNode::LatexFragment(inner) => *inner,
+            AstNode::ExportSnippet(inner) => *inner,
+            AstNode::FootnoteReference(inner) => *inner,
+            AstNode::Citation(inner) => *inner,
+            AstNode::CitationReference(inner) => *inner,
+            AstNode::InlineBabelCall(inner) => *inner,
+            AstNode::InlineSourceBlock(inner) => *inner,
+            AstNode::LineBreak(inner) => *inner,
+            AstNode::Target(inner) => *inner,
+            AstNode::StatisticsCookie(inner) => *inner,
+            AstNode::Subscript(inner) => *inner,
+            AstNode::Superscript(inner) => *inner,
+            AstNode::TableCell(inner) => *inner,
+            AstNode::Timestamp(inner) => *inner,
+        }
+    }
+}
+
+impl<'s> GetElispFact<'s> for Element<'s> {
+    fn get_elisp_fact<'b>(&'b self) -> &'b dyn ElispFact<'s> {
+        match self {
+            Element::Paragraph(inner) => inner,
+            Element::PlainList(inner) => inner,
+            Element::CenterBlock(inner) => inner,
+            Element::QuoteBlock(inner) => inner,
+            Element::SpecialBlock(inner) => inner,
+            Element::DynamicBlock(inner) => inner,
+            Element::FootnoteDefinition(inner) => inner,
+            Element::Comment(inner) => inner,
+            Element::Drawer(inner) => inner,
+            Element::PropertyDrawer(inner) => inner,
+            Element::Table(inner) => inner,
+            Element::VerseBlock(inner) => inner,
+            Element::CommentBlock(inner) => inner,
+            Element::ExampleBlock(inner) => inner,
+            Element::ExportBlock(inner) => inner,
+            Element::SrcBlock(inner) => inner,
+            Element::Clock(inner) => inner,
+            Element::DiarySexp(inner) => inner,
+            Element::Planning(inner) => inner,
+            Element::FixedWidthArea(inner) => inner,
+            Element::HorizontalRule(inner) => inner,
+            Element::Keyword(inner) => inner,
+            Element::BabelCall(inner) => inner,
+            Element::LatexEnvironment(inner) => inner,
+        }
+    }
+}
+
+impl<'s> GetElispFact<'s> for Object<'s> {
+    fn get_elisp_fact<'b>(&'b self) -> &'b dyn ElispFact<'s> {
+        match self {
+            Object::Bold(inner) => inner,
+            Object::Italic(inner) => inner,
+            Object::Underline(inner) => inner,
+            Object::StrikeThrough(inner) => inner,
+            Object::Code(inner) => inner,
+            Object::Verbatim(inner) => inner,
+            Object::PlainText(inner) => inner,
+            Object::RegularLink(inner) => inner,
+            Object::RadioLink(inner) => inner,
+            Object::RadioTarget(inner) => inner,
+            Object::PlainLink(inner) => inner,
+            Object::AngleLink(inner) => inner,
+            Object::OrgMacro(inner) => inner,
+            Object::Entity(inner) => inner,
+            Object::LatexFragment(inner) => inner,
+            Object::ExportSnippet(inner) => inner,
+            Object::FootnoteReference(inner) => inner,
+            Object::Citation(inner) => inner,
+            Object::CitationReference(inner) => inner,
+            Object::InlineBabelCall(inner) => inner,
+            Object::InlineSourceBlock(inner) => inner,
+            Object::LineBreak(inner) => inner,
+            Object::Target(inner) => inner,
+            Object::StatisticsCookie(inner) => inner,
+            Object::Subscript(inner) => inner,
+            Object::Superscript(inner) => inner,
+            Object::Timestamp(inner) => inner,
+        }
+    }
+}
+
+impl<'s> ElispFact<'s> for Document<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "org-data".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Section<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "section".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Heading<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "headline".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainList<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "plain-list".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainListItem<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "item".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for CenterBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "center-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for QuoteBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "quote-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for SpecialBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "special-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for DynamicBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "dynamic-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for FootnoteDefinition<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "footnote-definition".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Drawer<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "drawer".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PropertyDrawer<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "property-drawer".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for NodeProperty<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "node-property".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Table<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "table".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for TableRow<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "table-row".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Paragraph<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "paragraph".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for TableCell<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "table-cell".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Comment<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "comment".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for VerseBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "verse-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for CommentBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "comment-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for ExampleBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "example-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for ExportBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "export-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for SrcBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "src-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Clock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "clock".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for DiarySexp<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "diary-sexp".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Planning<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "planning".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for FixedWidthArea<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "fixed-width".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for HorizontalRule<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "horizontal-rule".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Keyword<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "keyword".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for BabelCall<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "babel-call".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for LatexEnvironment<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "latex-environment".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Bold<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "bold".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Italic<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "italic".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Underline<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "underline".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for StrikeThrough<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "strike-through".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Code<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "code".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Verbatim<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "verbatim".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for RegularLink<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for RadioLink<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for RadioTarget<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "radio-target".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainLink<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for AngleLink<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for OrgMacro<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "macro".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Entity<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "entity".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for LatexFragment<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "latex-fragment".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for ExportSnippet<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "export-snippet".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for FootnoteReference<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "footnote-reference".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Citation<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "citation".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for CitationReference<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "citation-reference".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for InlineBabelCall<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "inline-babel-call".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for InlineSourceBlock<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "inline-src-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for LineBreak<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "line-break".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Target<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "target".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for StatisticsCookie<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "statistics-cookie".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Subscript<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "subscript".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Superscript<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "superscript".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Timestamp<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        "timestamp".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainText<'s> {
+    fn get_elisp_name<'b>(&'b self) -> Cow<'s, str> {
+        // plain text from upstream emacs does not actually have a name but this is included here to make rendering the status diff easier.
+        "plain-text".into()
+    }
+}

src/compare/macros.rs

@@ -0,0 +1,155 @@
+/// Assert only the listed properties exist on the Emacs AST node and compare their values to the values on the rust AST node.
+///
+/// Example invocation:
+/// ```
+///     if let Some((new_status, new_message)) = compare_properties!(
+///         emacs,
+///         rust,
+///         (
+///             EmacsField::Required(":key"),
+///             |r| Some(r.key),
+///             compare_property_quoted_string
+///         ),
+///         (
+///             EmacsField::Required(":value"),
+///             |r| Some(r.contents),
+///             compare_property_quoted_string
+///         ),
+///         (EmacsField::Required(":pre-blank"), compare_identity, compare_noop)
+///     )? {
+///         this_status = new_status;
+///         message = new_message;
+///     }
+/// ```
+///
+/// Or if the node has no properties aside from :standard-properties, we can still assert that the node has no unexpected properties:
+/// ```
+///     if let Some((new_status, new_message)) = compare_properties!(emacs)? {
+///         this_status = new_status;
+///         message = new_message;
+///     }
+/// ```
+macro_rules! compare_properties {
+    ($emacs:expr, $rust:expr, $(($emacs_field:expr, $rust_value_getter:expr, $compare_fn: expr)),+) => {
+        {
+            let mut this_status = DiffStatus::Good;
+            let mut message: Option<String> = None;
+            let children = $emacs.as_list()?;
+            let attributes_child = children
+            .iter()
+            .nth(1)
+            .ok_or("Should have an attributes child.")?;
+            let attributes_map = attributes_child.as_map()?;
+            let mut emacs_keys: BTreeSet<&str> = attributes_map.keys().map(|s| *s).collect();
+            if emacs_keys.contains(":standard-properties") {
+                emacs_keys.remove(":standard-properties");
+            } else {
+                this_status = DiffStatus::Bad;
+                message = Some(format!(
+                    "Emacs token lacks :standard-properties field.",
+                ));
+            }
+            $(
+            match $emacs_field {
+                EmacsField::Required(name) if emacs_keys.contains(name) => {
+                    emacs_keys.remove(name);
+                },
+                EmacsField::Optional(name) if emacs_keys.contains(name) => {
+                    emacs_keys.remove(name);
+                },
+                EmacsField::Required(name) => {
+                    this_status = DiffStatus::Bad;
+                    message = Some(format!(
+                        "Emacs token lacks required field: {}",
+                        name
+                    ));
+                },
+                EmacsField::Optional(_name) => {},
+            }
+            )+
+
+            if !emacs_keys.is_empty() {
+                let unexpected_keys: Vec<&str> = emacs_keys.into_iter().collect();
+                let unexpected_keys = unexpected_keys.join(", ");
+                this_status = DiffStatus::Bad;
+                message = Some(format!(
+                    "Emacs token had extra field(s): {}",
+                    unexpected_keys
+                ));
+            }
+
+            $(
+            let emacs_name = match $emacs_field {
+                EmacsField::Required(name) => {
+                    name
+                },
+                EmacsField::Optional(name) => {
+                    name
+                },
+            };
+            let result = $compare_fn($emacs, $rust, emacs_name, $rust_value_getter)?;
+            match result {
+                Some((DiffStatus::Good, _)) => unreachable!("No comparison functions should return Some() when DiffStatus is good."),
+                Some((status, msg)) => {
+                    this_status = status;
+                    message = msg;
+                },
+                _ => {}
+            }
+            )+
+
+            match this_status {
+                DiffStatus::Good => {
+                    let result: Result<_, Box<dyn std::error::Error>> = Ok(None);
+                    result
+                },
+                _ => {
+                    Ok(Some((this_status, message)))
+                }
+            }
+        }
+    };
+    // Default case for when there are no expected properties except for :standard-properties
+    ($emacs:expr) => {
+        {
+            let mut this_status = DiffStatus::Good;
+            let mut message: Option<String> = None;
+            let children = $emacs.as_list()?;
+            let attributes_child = children
+            .iter()
+            .nth(1)
+            .ok_or("Should have an attributes child.")?;
+            let attributes_map = attributes_child.as_map()?;
+            let mut emacs_keys: BTreeSet<&str> = attributes_map.keys().map(|s| *s).collect();
+            if emacs_keys.contains(":standard-properties") {
+                emacs_keys.remove(":standard-properties");
+            } else {
+                this_status = DiffStatus::Bad;
+                message = Some(format!(
+                    "Emacs token lacks :standard-properties field.",
+                ));
+            }
+
+            if !emacs_keys.is_empty() {
+                let unexpected_keys: Vec<&str> = emacs_keys.into_iter().collect();
+                let unexpected_keys = unexpected_keys.join(", ");
+                this_status = DiffStatus::Bad;
+                message = Some(format!(
+                    "Emacs token had extra field(s): {}",
+                    unexpected_keys
+                ));
+            }
+            match this_status {
+                DiffStatus::Good => {
+                    let result: Result<_, Box<dyn std::error::Error>> = Ok(None);
+                    result
+                },
+                _ => {
+                    Ok(Some((this_status, message)))
+                }
+            }
+        }
+    };
+}
+
+pub(crate) use compare_properties;

src/compare/mod.rs

@@ -1,7 +1,12 @@
 mod compare;
+mod compare_field;
 mod diff;
+mod elisp_fact;
+mod macros;
 mod parse;
 mod sexp;
 mod util;
 pub use compare::run_anonymous_compare;
+pub use compare::run_anonymous_compare_with_settings;
 pub use compare::run_compare_on_file;
+pub use compare::run_compare_on_file_with_settings;

src/compare/parse.rs

@@ -1,8 +1,33 @@
 use std::path::Path;
 use std::process::Command;
 
-pub fn emacs_parse_anonymous_org_document<C>(
+use crate::context::HeadlineLevelFilter;
+use crate::settings::GlobalSettings;
+
+/// Generate elisp to configure org-mode parsing settings
+///
+/// Currently only org-list-allow-alphabetical is supported.
+fn global_settings_elisp(global_settings: &GlobalSettings) -> String {
+    // This string concatenation is wildly inefficient but its only called in tests 🤷.
+    let mut ret = "".to_owned();
+    if global_settings.list_allow_alphabetical {
+        ret += "(setq org-list-allow-alphabetical t)\n"
+    }
+    if global_settings.tab_width != crate::settings::DEFAULT_TAB_WIDTH {
+        ret += format!("(setq-default tab-width {})", global_settings.tab_width).as_str();
+    }
+    if global_settings.odd_levels_only != HeadlineLevelFilter::default() {
+        ret += match global_settings.odd_levels_only {
+            HeadlineLevelFilter::Odd => "(setq org-odd-levels-only t)\n",
+            HeadlineLevelFilter::OddEven => "(setq org-odd-levels-only nil)\n",
+        };
+    }
+    ret
+}
+
+pub(crate) fn emacs_parse_anonymous_org_document<C>(
     file_contents: C,
+    global_settings: &GlobalSettings,
 ) -> Result<String, Box<dyn std::error::Error>>
 where
     C: AsRef<str>,
@@ -14,10 +39,12 @@ where
      (require 'org)
      (defun org-table-align () t)
      (insert "{escaped_file_contents}")
+     {global_settings}
      (org-mode)
      (message "%s" (pp-to-string (org-element-parse-buffer)))
 )"#,
-        escaped_file_contents = escaped_file_contents
+        escaped_file_contents = escaped_file_contents,
+        global_settings = global_settings_elisp(global_settings)
     );
     let mut cmd = Command::new("emacs");
     let cmd = cmd
@@ -33,7 +60,10 @@ where
     Ok(String::from_utf8(org_sexp)?)
 }
 
-pub fn emacs_parse_file_org_document<P>(file_path: P) -> Result<String, Box<dyn std::error::Error>>
+pub(crate) fn emacs_parse_file_org_document<P>(
+    file_path: P,
+    global_settings: &GlobalSettings,
+) -> Result<String, Box<dyn std::error::Error>>
 where
     P: AsRef<Path>,
 {
@@ -46,9 +76,17 @@ where
         r#"(progn
      (require 'org)
      (defun org-table-align () t)
+     (setq vc-handled-backends nil)
+     {global_settings}
+     (find-file-read-only "{file_path}")
      (org-mode)
      (message "%s" (pp-to-string (org-element-parse-buffer)))
-)"#
+)"#,
+        global_settings = global_settings_elisp(global_settings),
+        file_path = file_path
+            .as_os_str()
+            .to_str()
+            .expect("File name should be valid utf-8.")
     );
     let mut cmd = Command::new("emacs");
     let cmd = cmd
@@ -57,8 +95,6 @@ where
         .arg("--no-site-file")
         .arg("--no-splash")
         .arg("--batch")
-        .arg("--insert")
-        .arg(file_path.as_os_str())
         .arg("--eval")
         .arg(elisp_script);
     let out = cmd.output()?;

src/compare/sexp.rs

@@ -1,9 +1,10 @@
 use std::collections::HashMap;
 
 use nom::branch::alt;
-use nom::bytes::complete::escaped;
 use nom::bytes::complete::tag;
 use nom::bytes::complete::take_till1;
+use nom::character::complete::anychar;
+use nom::character::complete::digit1;
 use nom::character::complete::multispace0;
 use nom::character::complete::multispace1;
 use nom::character::complete::one_of;
@@ -11,6 +12,7 @@ use nom::combinator::map;
 use nom::combinator::not;
 use nom::combinator::opt;
 use nom::combinator::peek;
+use nom::combinator::recognize;
 use nom::multi::separated_list1;
 use nom::sequence::delimited;
 use nom::sequence::preceded;
@@ -18,6 +20,8 @@ use nom::sequence::tuple;
 
 use crate::error::Res;
 
+const MAX_OCTAL_LENGTH: usize = 3;
+
 #[derive(Debug)]
 pub enum Token<'s> {
     Atom(&'s str),
@@ -35,6 +39,7 @@ pub struct TextWithProperties<'s> {
 enum ParseState {
     Normal,
     Escape,
+    Octal(Vec<u8>),
 }
 
 impl<'s> Token<'s> {
@@ -106,8 +111,8 @@ fn is_slice_of(parent: &str, child: &str) -> bool {
 }
 
 /// Get a slice of the string that was consumed in a parser using the original input to the parser and the remaining input after the parser.
-pub fn get_consumed<'s>(input: &'s str, remaining: &'s str) -> &'s str {
-    assert!(is_slice_of(input, remaining));
+fn get_consumed<'s>(input: &'s str, remaining: &'s str) -> &'s str {
+    debug_assert!(is_slice_of(input, remaining));
     let source = {
         let offset = remaining.as_ptr() as usize - input.as_ptr() as usize;
         &input[..offset]
@@ -116,7 +121,7 @@ pub fn get_consumed<'s>(input: &'s str, remaining: &'s str) -> &'s str {
 }
 
 pub(crate) fn unquote(text: &str) -> Result<String, Box<dyn std::error::Error>> {
-    let mut out = String::with_capacity(text.len());
+    let mut out: Vec<u8> = Vec::with_capacity(text.len());
     if !text.starts_with(r#"""#) {
         return Err("Quoted text does not start with quote.".into());
     }
@@ -125,30 +130,53 @@ pub(crate) fn unquote(text: &str) -> Result<String, Box<dyn std::error::Error>>
     }
     let interior_text = &text[1..(text.len() - 1)];
     let mut state = ParseState::Normal;
-    for current_char in interior_text.chars().into_iter() {
+    for current_char in interior_text.bytes().into_iter() {
+        // Check to see if octal finished
         state = match (state, current_char) {
-            (ParseState::Normal, '\\') => ParseState::Escape,
+            (ParseState::Octal(octal), b'0'..=b'7') if octal.len() < MAX_OCTAL_LENGTH => {
+                ParseState::Octal(octal)
+            }
+            (ParseState::Octal(octal), _) => {
+                let octal_number_string = String::from_utf8(octal)?;
+                let decoded_byte = u8::from_str_radix(&octal_number_string, 8)?;
+                out.push(decoded_byte);
+                ParseState::Normal
+            }
+            (state, _) => state,
+        };
+
+        state = match (state, current_char) {
+            (ParseState::Normal, b'\\') => ParseState::Escape,
             (ParseState::Normal, _) => {
                 out.push(current_char);
                 ParseState::Normal
             }
-            (ParseState::Escape, 'n') => {
-                out.push('\n');
+            (ParseState::Escape, b'n') => {
+                out.push(b'\n');
                 ParseState::Normal
             }
-            (ParseState::Escape, '\\') => {
-                out.push('\\');
+            (ParseState::Escape, b'\\') => {
+                out.push(b'\\');
                 ParseState::Normal
             }
-            (ParseState::Escape, '"') => {
-                out.push('"');
+            (ParseState::Escape, b'"') => {
+                out.push(b'"');
                 ParseState::Normal
             }
-            _ => todo!(),
+            (ParseState::Escape, b'0'..=b'7') => {
+                let mut octal = Vec::with_capacity(MAX_OCTAL_LENGTH);
+                octal.push(current_char);
+                ParseState::Octal(octal)
+            }
+            (ParseState::Octal(mut octal), b'0'..=b'7') => {
+                octal.push(current_char);
+                ParseState::Octal(octal)
+            }
+            _ => panic!("Invalid state unquoting string."),
         };
     }
 
-    Ok(out)
+    Ok(String::from_utf8(out)?)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -210,15 +238,30 @@ fn unquoted_atom<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
 fn quoted_atom<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
-    let (remaining, _) = tag(r#"""#)(input)?;
-    let (remaining, _) = escaped(
-        take_till1(|c| match c {
-            '\\' | '"' => true,
-            _ => false,
-        }),
-        '\\',
-        one_of(r#""n\\"#),
-    )(remaining)?;
+    let (mut remaining, _) = tag(r#"""#)(input)?;
+    let mut in_escape = false;
+    loop {
+        if in_escape {
+            let (remain, _) = alt((recognize(one_of(r#""n\\"#)), digit1))(remaining)?;
+            remaining = remain;
+            in_escape = false;
+        } else {
+            let end_quote = tag::<_, _, nom::error::Error<_>>(r#"""#)(remaining);
+            if end_quote.is_ok() {
+                break;
+            }
+
+            let escape_backslash = tag::<_, _, nom::error::Error<_>>("\\")(remaining);
+            if let Ok((remain, _)) = escape_backslash {
+                remaining = remain;
+                in_escape = true;
+                continue;
+            }
+
+            let (remain, _) = anychar(remaining)?;
+            remaining = remain;
+        }
+    }
     let (remaining, _) = tag(r#"""#)(remaining)?;
     let source = get_consumed(input, remaining);
     Ok((remaining, Token::Atom(source.into())))