Merge branch 'reduce_heap_iter'

Hopefully most types won't need so much care.

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "organic"
-version = "0.1.6"
+version = "0.1.8"
 authors = ["Tom Alexander <tom@fizz.buzz>"]
 description = "An org-mode parser."
 edition = "2021"
@@ -21,9 +21,15 @@ name = "organic"
 path = "src/lib.rs"
 
 [[bin]]
-# This bin exists for development purposes only. The real target of this crate is the library.
-name = "parse"
-path = "src/main.rs"
+  # This bin exists for development purposes only. The real target of this crate is the library.
+  name = "parse"
+  path = "src/main.rs"
+
+[[bin]]
+  # This bin exists for development purposes only. The real target of this crate is the library.
+  name = "compare"
+  path = "src/bin_compare.rs"
+  required-features = ["compare"]
 
 [dependencies]
 nom = "7.1.1"

Makefile

@@ -33,6 +33,10 @@ release:
 clean:
 > cargo clean
 
+.PHONY: format
+format:
+> $(MAKE) -C docker/cargo_fmt run
+
 .PHONY: test
 test:
 > cargo test --no-default-features --features compare --no-fail-fast --lib --test test_loader -- --test-threads $(TESTJOBS)

README.md

@@ -2,12 +2,84 @@
 
 Organic is an emacs-less implementation of an [org-mode](https://orgmode.org/) parser.
 
-
 ## Project Status
 
-This project is a personal learning project to grow my experience in [rust](https://www.rust-lang.org/). It is under development and at this time I would not recommend anyone use this code. The goal is to turn this into a project others can use, at which point more information will appear in this README.
+This project is still under HEAVY development. While the version remains v0.1.x the API will be changing often. Once we hit v0.2.x we will start following semver.
 
+Currently, the parser is able to correctly identify the start/end bounds of all the org-mode objects and elements (except table.el tables, org-mode tables are supported) but many of the interior properties are not yet populated.
+
+### Project Goals
+- We aim to provide perfect parity with the emacs org-mode parser. In that regard, any document that parses differently between Emacs and Organic is considered a bug.
+- The parser should be fast. We're not doing anything special, but since this is written in Rust and natively compiled we should be able to beat the existing parsers.
+- The parser should have minimal dependencies. This should reduce effort w.r.t.: security audits, legal compliance, portability.
+- The parser should be usable everywhere. In the interest of getting org-mode used in as many places as possible, this parser should be usable by everyone everywhere. This means:
+  - It must have a permissive license for use in proprietary code bases.
+  - We will investigate compiling to WASM. This is an important goal of the project and will definitely happen, but only after the parser has a more stable API.
+  - We will investigate compiling to a C library for native linking to other code. This is more of a maybe-goal for the project.
+### Project Non-Goals
+- This project will not include an elisp engine since that would drastically increase the complexity of the code. Any features requiring an elisp engine will not be implemented (for example, Emacs supports embedded eval expressions in documents but this parser will never support that).
+- This project is exclusively an org-mode **parser**. This limits its scope to roughly the output of `(org-element-parse-buffer)`. It will not render org-mode documents in other formats like HTML or LaTeX.
+### Project Maybe-Goals
+- table.el support. Currently we support org-mode tables but org-mode also allows table.el tables. So far, their use in org-mode documents seems rather uncommon so this is a low-priority feature.
+- Document editing support. I do not anticipate any advanced editing features to make editing ergonomic, but it should be relatively easy to be able to parse an org-mode document and serialize it back into org-mode. This would enable cool features to be built on top of the library like auto-formatters. To accomplish this feature, We'd have to capture all of the various separators and whitespace that we are currently simply throwing away. This would add many additional fields to the parsed structs and it would add more noise to the parsers themselves, so I do not want to approach this feature until the parser is more complete since it would make modifications and refactoring more difficult.
+### Supported Versions
+This project targets the version of Emacs and Org-mode that are built into the [organic-test docker image](docker/organic_test/Dockerfile). This is newer than the version of Org-mode that shipped with Emacs 29.1. The parser itself does not depend on Emacs or Org-mode though, so this only matters for development purposes when running the automated tests that compare against upstream Org-mode.
+
+## Using this library
+TODO: Add section on using Organic as a library (which is the intended use for this project). This will be added when we have a bit more API stability since currently the library is under heavy development.
+
+## Development
+
+### The parse binary
+This program takes org-mode input either streamed in on stdin or as paths to files passed in as arguments. It then parses them using Organic and dumps the result to stdout. This program is intended solely as a development tool. Examples:
+```bash
+cat /foo/bar.org | cargo run --bin parse
+```
+```bash
+cargo build --profile release-lto
+./target/release-lto/parse /foo/bar.org /lorem/ipsum.org
+```
+
+### The compare binary
+This program takes org-mode input either streamed in on stdin or as paths to files passed in as arguments. It then parses them using Organic and the official Emacs Org-mode parser and compares the parse result. This program is intended solely as a development tool. Since org-mode is a moving target, it is recommended that you run this through docker since we pin the version of org-mode to a specific revision. Examples:
+```bash
+cat /foo/bar.org | ./scripts/run_docker_compare.bash
+```
+```bash
+./scripts/run_docker_compare.bash /foo/bar.org /lorem/ipsum.org
+```
+
+Not recommended since it is not through docker:
+
+```bash
+cat /foo/bar.org | cargo run --features compare --bin compare
+```
+```bash
+cargo build --profile release-lto --features compare
+./target/release-lto/compare /foo/bar.org /lorem/ipsum.org
+```
+
+## Running the tests
+There are three levels of tests for this repository: the standard tests, the autogenerated tests, and the foreign document tests.
+
+### The standard tests
+These are regular hand-written rust tests. These can be run with:
+```bash
+make unittest
+```
+
+### The auto-generated tests
+These tests are automatically generated from the files in the `org_mode_samples` directory and they are still integrated with the rust/cargo testing framework. For each org-mode document in that folder, a test is generated that will parse the document with both Organic and the official Emacs Org-mode parser and then it will compare the parse results. Any deviation is considered a failure. Since org-mode is a moving target, it is recommended that you run these tests inside docker since the `organic-test` docker image is pinned to a specific revision of org-mode. These can be run with:
+```bash
+make dockertest
+```
+
+### The foreign document tests
+These tests function the same as the auto-generated tests except they are **not** integrated with the rust/cargo testing framework and they involve comparing the parse of org-mode documents that live outside this repository. This allows us to test against a far greater variety of org-mode input documents without pulling massive sets of org-mode documents into this repository. The recommended way to run these tests is still through docker because it pins org-mode and the test documents to specific git revisions. These can be run with:
+```bash
+make foreign_document_test
+```
 
 ## License
 
-This project is released under the public-domain-equivalent [0BSD license](https://www.tldrlegal.com/license/bsd-0-clause-license). This license puts no restrictions on the use of this code (you do not even have to include the copyright notice or license text when using it). HOWEVER, this project has a couple permissively licensed dependencies which do require their copyright notices and/or license texts to be included. I am not a lawyer and this is not legal advice but it is my layperson's understanding that if you distribute a binary with this library linked in, you will need to abide by their terms since their code will also be linked in your binary. I try to keep the dependencies to a minimum and the most restrictive dependency I will ever include is a permissively licensed one.
+This project is released under the public-domain-equivalent [0BSD license](https://www.tldrlegal.com/license/bsd-0-clause-license), however, this project has a couple permissively licensed non-public-domain-equivalent dependencies which require their copyright notices and/or license texts to be included. I am not a lawyer and this is not legal advice but it is my layperson's understanding that if you distribute a binary statically linking this library, you will need to abide by their terms since their code will also be linked in your binary.

build.rs

@@ -16,6 +16,9 @@ fn main() {
     let destination = Path::new(&out_dir).join("tests.rs");
     let mut test_file = File::create(&destination).unwrap();
 
+    // Re-generate the tests if any org-mode files change
+    println!("cargo:rerun-if-changed=org_mode_samples");
+
     write_header(&mut test_file);
 
     let test_files = WalkDir::new("org_mode_samples")
@@ -71,10 +74,6 @@ fn write_header(test_file: &mut File) {
         test_file,
         r#"
 #[feature(exit_status_error)]
-use organic::compare_document;
-use organic::parser::parse;
-use organic::emacs_parse_anonymous_org_document;
-use organic::parser::sexp::sexp_with_padding;
 
 "#
     )
@@ -86,7 +85,6 @@ fn is_expect_fail(name: &str) -> Option<&str> {
     match name {
         "autogen_greater_element_drawer_drawer_with_headline_inside" => Some("Apparently lines with :end: become their own paragraph. This odd behavior needs to be investigated more."),
         "autogen_element_container_priority_footnote_definition_dynamic_block" => Some("Apparently broken begin lines become their own paragraph."),
-        "autogen_lesser_element_paragraphs_paragraph_with_backslash_line_breaks" => Some("The text we're getting out of the parse tree is already processed to remove line breaks, so our comparison needs to take that into account."),
         _ => None,
     }
 }

docker/organic_test/Dockerfile

@@ -1,5 +1,5 @@
 FROM alpine:3.17 AS build
-RUN apk add --no-cache build-base musl-dev git autoconf make texinfo gnutls-dev ncurses-dev gawk
+RUN apk add --no-cache build-base musl-dev git autoconf make texinfo gnutls-dev ncurses-dev gawk libgccjit-dev
 
 
 FROM build AS build-emacs
@@ -8,13 +8,13 @@ RUN git clone --depth 1 --branch $EMACS_VERSION https://git.savannah.gnu.org/git
 WORKDIR /root/emacs
 RUN mkdir /root/dist
 RUN ./autogen.sh
-RUN ./configure --prefix /usr --without-x --without-sound
+RUN ./configure --prefix /usr --without-x --without-sound --with-native-compilation=aot
 RUN make
 RUN make DESTDIR="/root/dist" install
 
 
 FROM build AS build-org-mode
-ARG ORG_VERSION=163bafb43dcc2bc94a2c7ccaa77d3d1dd488f1af
+ARG ORG_VERSION=c703541ffcc14965e3567f928de1683a1c1e33f6
 COPY --from=build-emacs /root/dist/ /
 RUN mkdir /root/dist
 # Savannah does not allow fetching specific revisions, so we're going to have to put unnecessary load on their server by cloning main and then checking out the revision we want.
@@ -27,7 +27,7 @@ RUN make DESTDIR="/root/dist" install
 
 FROM rustlang/rust:nightly-alpine3.17 AS tester
 ENV LANG=en_US.UTF-8
-RUN apk add --no-cache musl-dev ncurses gnutls
+RUN apk add --no-cache musl-dev ncurses gnutls libgccjit
 RUN cargo install --locked --no-default-features --features ci-autoclean cargo-cache
 COPY --from=build-emacs /root/dist/ /
 COPY --from=build-org-mode /root/dist/ /
@@ -88,14 +88,20 @@ ARG DOOMEMACS_PATH=/foreign_documents/doomemacs
 ARG DOOMEMACS_REPO=https://github.com/doomemacs/doomemacs.git
 RUN mkdir -p $DOOMEMACS_PATH && git -C $DOOMEMACS_PATH init --initial-branch=main && git -C $DOOMEMACS_PATH remote add origin $DOOMEMACS_REPO && git -C $DOOMEMACS_PATH fetch origin $DOOMEMACS_VERSION && git -C $DOOMEMACS_PATH checkout FETCH_HEAD
 
+ARG WORG_VERSION=0c8d5679b536af450b61812246a3e02b8103f4b8
+ARG WORG_PATH=/foreign_documents/worg
+ARG WORG_REPO=https://git.sr.ht/~bzg/worg
+RUN mkdir -p $WORG_PATH && git -C $WORG_PATH init --initial-branch=main && git -C $WORG_PATH remote add origin $WORG_REPO && git -C $WORG_PATH fetch origin $WORG_VERSION && git -C $WORG_PATH checkout FETCH_HEAD
+
 
 FROM tester as foreign-document-test
 RUN apk add --no-cache bash coreutils
 RUN mkdir /foreign_documents
-COPY --from=build-org-mode /root/org-mode /foreign_documents/org-mode
-COPY --from=build-emacs /root/emacs /foreign_documents/emacs
 COPY --from=foreign-document-gather /foreign_documents/howardabrams /foreign_documents/howardabrams
 COPY --from=foreign-document-gather /foreign_documents/doomemacs /foreign_documents/doomemacs
+COPY --from=foreign-document-gather /foreign_documents/worg /foreign_documents/worg
+COPY --from=build-org-mode /root/org-mode /foreign_documents/org-mode
+COPY --from=build-emacs /root/emacs /foreign_documents/emacs
 COPY foreign_document_test_entrypoint.sh /entrypoint.sh
 RUN chmod +x /entrypoint.sh
 ENTRYPOINT ["/entrypoint.sh"]

docker/organic_test/foreign_document_test_entrypoint.sh

@@ -23,7 +23,7 @@ function main {
     if [ "${CARGO_TARGET_DIR:-}" = "" ]; then
         CARGO_TARGET_DIR=$(realpath target/)
     fi
-    PARSE="${CARGO_TARGET_DIR}/release-lto/parse"
+    PARSE="${CARGO_TARGET_DIR}/release-lto/compare"
 
     local all_status=0
     set +e
@@ -32,6 +32,8 @@ function main {
     if [ "$?" -ne 0 ]; then all_status=1; fi
     (run_compare_function "emacs" compare_all_org_document "/foreign_documents/emacs")
     if [ "$?" -ne 0 ]; then all_status=1; fi
+    (run_compare_function "worg" compare_all_org_document "/foreign_documents/worg")
+    if [ "$?" -ne 0 ]; then all_status=1; fi
     (run_compare_function "howard_abrams" compare_howard_abrams)
     if [ "$?" -ne 0 ]; then all_status=1; fi
     (run_compare_function "doomemacs" compare_all_org_document "/foreign_documents/doomemacs")
@@ -39,9 +41,9 @@ function main {
 
     set -e
     if [ "$all_status" -ne 0 ]; then
-        echo "$(red_text "Some tests failed.")"
+        red_text "Some tests failed."
     else
-        echo "$(green_text "All tests passed.")"
+        green_text "All tests passed."
     fi
     return "$all_status"
 }
@@ -62,8 +64,9 @@ function indent {
     local depth="$1"
     local scaled_depth=$((depth * 2))
     shift 1
-    local prefix=$(printf -- "%${scaled_depth}s")
-    while read l; do
+    local prefix
+    prefix=$(printf -- "%${scaled_depth}s")
+    while read -r l; do
         (IFS=' '; printf -- '%s%s\n' "$prefix" "$l")
     done
 }
@@ -91,12 +94,13 @@ function compare_all_org_document {
     local target_document
     local all_status=0
     while read target_document; do
-        local relative_path=$($REALPATH --relative-to "$root_dir" "$target_document")
+        local relative_path
+        relative_path=$($REALPATH --relative-to "$root_dir" "$target_document")
         set +e
         (run_compare "$relative_path" "$target_document")
         if [ "$?" -ne 0 ]; then all_status=1; fi
         set -e
-    done<<<$(find "$root_dir" -type f -iname '*.org')
+    done<<<"$(find "$root_dir" -type f -iname '*.org' | sort)"
     return "$all_status"
 }
 

notes/optimization_ideas.org

@@ -25,3 +25,4 @@ This could significantly reduce our calls to exit matchers.
 I think targets would break this.
 
 The exit matchers are already implicitly building this behavior since they should all exit very early when the starting character is wrong. Putting this logic in a centralized place, far away from where those characters are actually going to be used, is unfortunate for readability.
+** Use exit matcher to cut off trailing whitespace instead of re-matching in plain lists.

org_mode_samples/greater_element/dynamic_block/no_closing_colon.org

@@ -0,0 +1,3 @@
+#+BEGIN: timestamp :format "%Y-%m-%d %H:%M"
+
+#+END

org_mode_samples/greater_element/greater_block/quote_block_with_leading_blank_line.org

@@ -0,0 +1,5 @@
+#+begin_quote
+
+foo
+
+#+end_quote

org_mode_samples/greater_element/plain_list/alphabetical_bullets.org

@@ -0,0 +1,3 @@
+# These are only allowed by configuring org-list-allow-alphabetical which the automated tests are not currently set up to do, so this will parse as a paragraph:
+a. foo
+b. bar

org_mode_samples/greater_element/plain_list/description_list_empty_value.org

@@ -0,0 +1,6 @@
+- foo ::
+
+- bar ::
+
+
+baz

org_mode_samples/greater_element/plain_list/description_list_with_double_colon_in_tag.org

@@ -1 +1,2 @@
 - =foo :: bar= :: baz
+- lorem :: ipsum :: dolar

org_mode_samples/greater_element/plain_list/mixed_types.org

@@ -0,0 +1,3 @@
+1. foo
+- bar
+- lorem :: ipsum

org_mode_samples/greater_element/plain_list/ordered_list_with_fake_tag.org

@@ -0,0 +1,2 @@
+# Since this is an ordered list, the text before the " :: " is NOT parsed as a tag.
+1. foo :: bar

org_mode_samples/greater_element/table/align_unaligned_table.org

@@ -0,0 +1,6 @@
+# The STARTUP directive here instructs org-mode to align tables which emacs normally does when opening the file. Since Organic is solely a parser, we have no business editing the org-mode document so Organic does not handle aligning tables, so in order for this test to pass, we have to avoid that behavior in Emacs.
+#+STARTUP: align
+
+|foo|bar|
+|-
+|lorem|ipsum|

org_mode_samples/greater_element/table/empty_formula.org

@@ -0,0 +1,4 @@
+| Name | Value |
+|------+-------|
+| foo  | bar   |
+#+tblfm:

org_mode_samples/lesser_element/diary_sexp/not_real_sexp.org

@@ -0,0 +1,6 @@
+%%(foo
+)
+
+%%(bar    ;  baz
+
+lorem

org_mode_samples/lesser_element/fixed_width_area/tab_instead_of_space.org

@@ -0,0 +1,2 @@
+# Fixed width areas must begin with colon followed by a space, not a tab, so this is not a fixed width area.
+:	foo

org_mode_samples/object/plain_link/with_parenthesis.org

@@ -0,0 +1,11 @@
+# Should be a link:
+https://en.wikipedia.org/wiki/Shebang_(Unix)
+
+# No closing parenthesis, so link ends at underscore.
+https://en.wikipedia.org/wiki/Shebang_(Unix
+
+# Parenthesis only allowed to depth of 2 so link ends at underscore.
+https://en.wikipedia.org/wiki/Shebang_(((Unix)))
+
+# Even though they eventually become balanced, we hit negative parenthesis depth so link ends at )
+https://en.wikipedia.org/wiki/Shebang)Unix(

org_mode_samples/object/radio_link/different_case.org

@@ -0,0 +1,3 @@
+<<<Foo Bar Baz>>>
+
+foo bar baz

org_mode_samples/object/radio_link/different_whitespace.org

@@ -0,0 +1,6 @@
+<<<foo bar baz>>>
+
+
+foo
+bar
+baz

org_mode_samples/object/regular_link/elisp.org

@@ -0,0 +1 @@
+[[elisp:(local-set-key "\M-\x" 'foo-bar-baz)]]

org_mode_samples/object/regular_link/with_parenthesis.org

@@ -0,0 +1 @@
+[[https://en.wikipedia.org/wiki/Shebang_(Unix)]]

org_mode_samples/object/regular_link/wrapped_with_brackets.org

@@ -0,0 +1 @@
+[[[http://foo.bar/baz][lorem]]]

org_mode_samples/object/subscript_and_superscript/options_require_braces.org

@@ -0,0 +1,7 @@
+# Even though *exporting* honors the setting to require braces for subscript/superscript, the official org-mode parser still parses subscripts and superscripts.
+
+#+OPTIONS:    ^:{}
+foo_this isn't a subscript when exported due to lack of braces (but its still a subscript during parsing)
+
+
+bar_{this is a subscript}

org_mode_samples/object/subscript_and_superscript/with_parenthesis.org

@@ -0,0 +1,13 @@
+foo_(bar)
+
+foo_(b(ar)
+
+foo_(b{ar)
+
+foo_{b(ar}
+
+foo_(b(a)r)
+
+foo_b(a)r
+
+foo_(b+ar)

org_mode_samples/object/text_markup/double_star.org

@@ -0,0 +1 @@
+foo ** bar ** baz

org_mode_samples/object/text_markup/double_tilde.org

@@ -0,0 +1 @@
+foo ~~ bar ~~ baz

org_mode_samples/object/text_markup/target_substring.org

@@ -0,0 +1,4 @@
+# Since "foos" has an extra "s", this does not match the target.
+the foos bar
+
+The <<<foo>>> and stuff.

org_mode_samples/sections_and_headings/empty_heading.org

@@ -0,0 +1,2 @@
+* DONE
+* 

org_mode_samples/sections_and_headings/fast_access_todo_states.org

@@ -0,0 +1,6 @@
+#+TODO: TODO(t) INPROGRESS(i/!) | DONE(d!) CANCELED(c@/!)
+# !  : Log changes leading to this state.
+# @  : Log changes leading to this state and prompt for a comment to include.
+# /! : Log changes leaving this state if and only if to a state that does not log. This can be combined with the above like WAIT(w!/!) or DELAYED(d@/!)
+* INPROGRESS
+- State "TODO"       from "INPROGRESS" [2023-09-14 Thu 02:13]

org_mode_samples/sections_and_headings/odd_level_depth.org

@@ -0,0 +1,8 @@
+#+STARTUP: odd
+* Foo
+***** Bar
+* Baz
+*** Lorem
+* Ipsum
+**** Dolar
+***** Cat

scripts/run_docker_compare.bash

@@ -8,10 +8,26 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
 : ${TRACE:="NO"} # or YES to send traces to jaeger
 : ${BACKTRACE:="NO"} # or YES to print a rust backtrace when panicking
 : ${NO_COLOR:=""} # Set to anything to disable color output
+: ${PROFILE:="debug"}
 
 REALPATH=$(command -v uu-realpath || command -v realpath)
 MAKE=$(command -v gmake || command -v make)
 
+############## Setup #########################
+
+function die {
+    local status_code="$1"
+    shift
+    (>&2 echo "${@}")
+    exit "$status_code"
+}
+
+function log {
+    (>&2 echo "${@}")
+}
+
+############## Program #########################
+
 function main {
     build_container
     launch_container "${@}"
@@ -23,7 +39,6 @@ function build_container {
 
 function launch_container {
     local additional_flags=()
-    local additional_args=()
     local features=(compare)
 
     if [ "$NO_COLOR" != "" ]; then
@@ -37,11 +52,8 @@ function launch_container {
     fi
 
     if [ "$SHELL" != "YES" ]; then
-        local features_joined=$(IFS=","; echo "${features[*]}")
-        additional_args+=(cargo run --no-default-features --features "$features_joined")
         additional_flags+=(--read-only)
     else
-        additional_args+=(/bin/sh)
         additional_flags+=(-t)
     fi
 
@@ -49,16 +61,50 @@ function launch_container {
         additional_flags+=(--env RUST_BACKTRACE=full)
     fi
 
+    if [ "$SHELL" = "YES" ]; then
+        exec docker run "${additional_flags[@]}" --init --rm -i --mount type=tmpfs,destination=/tmp -v "/:/input:ro" -v "$($REALPATH "$DIR/../"):/source:ro" --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source --entrypoint "" organic-test /bin/sh
+    fi
+
+    local features_joined
+    features_joined=$(IFS=","; echo "${features[*]}")
+
+    local build_flags=()
+    if [ "$PROFILE" = "dev" ] || [ "$PROFILE" = "debug" ]; then
+        PROFILE="debug"
+    else
+        build_flags+=(--profile "$PROFILE")
+    fi
+
+
     if [ $# -gt 0 ]; then
         # If we passed in args, we need to forward them along
         for path in "${@}"; do
-            local full_path=$($REALPATH "$path")
-            local containing_folder=$(dirname "$full_path")
-            local file_name=$(basename "$full_path")
-            docker run "${additional_flags[@]}" --init --rm -i --mount type=tmpfs,destination=/tmp -v "${containing_folder}:/input:ro"  -v "$($REALPATH "$DIR/../"):/source:ro" --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source --entrypoint "" organic-test "${additional_args[@]}" -- "/input/$file_name"
+            local full_path
+            full_path=$($REALPATH "$path")
+            init_script=$(cat <<EOF
+set -euo pipefail
+IFS=\$'\n\t'
+
+cargo build --bin compare --no-default-features --features "$features_joined" ${build_flags[@]}
+exec /target/${PROFILE}/compare "/input${full_path}"
+EOF
+                       )
+            docker run "${additional_flags[@]}" --init --rm -i --mount type=tmpfs,destination=/tmp -v "/:/input:ro"  -v "$($REALPATH "$DIR/../"):/source:ro" --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source --entrypoint "" organic-test sh -c "$init_script"
         done
     else
-        docker run "${additional_flags[@]}" --init --rm -i --mount type=tmpfs,destination=/tmp -v "$($REALPATH "$DIR/../"):/source:ro" --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source --entrypoint "" organic-test "${additional_args[@]}"
+        local current_directory init_script
+        current_directory=$(pwd)
+        init_script=$(cat <<EOF
+set -euo pipefail
+IFS=\$'\n\t'
+
+cargo build --bin compare --no-default-features --features "$features_joined" ${build_flags[@]}
+cd /input${current_directory}
+exec /target/${PROFILE}/compare
+EOF
+                       )
+
+        docker run "${additional_flags[@]}" --init --rm -i --mount type=tmpfs,destination=/tmp -v "/:/input:ro" -v "$($REALPATH "$DIR/../"):/source:ro" --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source --entrypoint "" organic-test sh -c "$init_script"
     fi
 }
 

scripts/run_docker_compare_bisect.bash

@@ -9,17 +9,6 @@ REALPATH=$(command -v uu-realpath || command -v realpath)
 
 ############## Setup #########################
 
-function cleanup {
-    for f in "${folders[@]}"; do
-        log "Deleting $f"
-        rm -rf "$f"
-    done
-}
-folders=()
-for sig in EXIT INT QUIT HUP TERM; do
-  trap "set +e; cleanup" "$sig"
-done
-
 function die {
     local status_code="$1"
     shift
@@ -34,18 +23,18 @@ function log {
 ############## Program #########################
 
 function main {
-    log "Is is recommended that the output of \`mktemp -d -t 'compare_bisect.XXXXXXXX'\` is inside a tmpfs filesystem since this script will make many writes to these folders."
-
-    local target_full_path=$($REALPATH "$1")
+    local target_full_path
+    target_full_path=$($REALPATH "$1")
     SOURCE_FOLDER=$(dirname "$target_full_path")
     TARGET_DOCUMENT=$(basename "$target_full_path")
 
 
     local good=0
-    local bad=$(wc -l "$SOURCE_FOLDER/$TARGET_DOCUMENT" | awk '{print $1}')
+    local bad
+    bad=$(wc -l "$SOURCE_FOLDER/$TARGET_DOCUMENT" | awk '{print $1}')
 
     set +e
-    run_parse "$bad" &> /dev/null
+    (run_parse "$bad")
     local status=$?
     set -e
     if [ $status -eq 0 ]; then
@@ -71,21 +60,12 @@ function main {
     echo "Bad line: $bad"
 }
 
-function setup_temp_dir {
-    local temp_dir=$(mktemp -d -t 'compare_bisect.XXXXXXXX')
-    cp -r "$SOURCE_FOLDER/"* "$temp_dir/"
-    echo "$temp_dir"
-}
-
 function run_parse {
     local lines="$1"
-    local temp_dir=$(setup_temp_dir)
-    folders+=("$temp_dir")
-    cat "$SOURCE_FOLDER/$TARGET_DOCUMENT" | head -n "$lines" > "$temp_dir/$TARGET_DOCUMENT"
-    "${DIR}/run_docker_compare.bash" "$temp_dir/$TARGET_DOCUMENT"
+
+    cd "$SOURCE_FOLDER"
+    head -n "$lines" "$SOURCE_FOLDER/$TARGET_DOCUMENT" | PROFILE=release-lto "${DIR}/run_docker_compare.bash"
     local status=$?
-    rm -rf "$temp_dir"
-    # TODO: Remove temp_dir from folders
     return "$status"
 }
 

src/bin_compare.rs

@@ -0,0 +1,52 @@
+#![feature(round_char_boundary)]
+#![feature(exact_size_is_empty)]
+use std::io::Read;
+
+use organic::compare::run_anonymous_compare;
+use organic::compare::run_compare_on_file;
+
+#[cfg(feature = "tracing")]
+use crate::init_tracing::init_telemetry;
+#[cfg(feature = "tracing")]
+use crate::init_tracing::shutdown_telemetry;
+#[cfg(feature = "tracing")]
+mod init_tracing;
+
+#[cfg(not(feature = "tracing"))]
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    main_body()
+}
+
+#[cfg(feature = "tracing")]
+fn main() -> Result<(), Box<dyn std::error::Error>> {
+    let rt = tokio::runtime::Runtime::new()?;
+    let result = rt.block_on(async {
+        init_telemetry()?;
+        let main_body_result = main_body();
+        shutdown_telemetry()?;
+        main_body_result
+    });
+    result
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn main_body() -> Result<(), Box<dyn std::error::Error>> {
+    let args = std::env::args().skip(1);
+    if args.is_empty() {
+        let org_contents = read_stdin_to_string()?;
+        run_anonymous_compare(org_contents)
+    } else {
+        for arg in args {
+            run_compare_on_file(arg)?
+        }
+        Ok(())
+    }
+}
+
+fn read_stdin_to_string() -> Result<String, Box<dyn std::error::Error>> {
+    let mut stdin_contents = String::new();
+    std::io::stdin()
+        .lock()
+        .read_to_string(&mut stdin_contents)?;
+    Ok(stdin_contents)
+}

src/compare/compare.rs

@@ -0,0 +1,77 @@
+use std::path::Path;
+
+use crate::compare::diff::compare_document;
+use crate::compare::parse::emacs_parse_anonymous_org_document;
+use crate::compare::parse::emacs_parse_file_org_document;
+use crate::compare::parse::get_emacs_version;
+use crate::compare::parse::get_org_mode_version;
+use crate::compare::sexp::sexp;
+use crate::parser::parse;
+use crate::parser::parse_with_settings;
+use crate::GlobalSettings;
+use crate::LocalFileAccessInterface;
+
+pub fn run_anonymous_compare<P: AsRef<str>>(
+    org_contents: P,
+) -> Result<(), Box<dyn std::error::Error>> {
+    // TODO: This is a work-around to pretend that dos line endings do not exist. It would be better to handle the difference in line endings.
+    let org_contents = org_contents.as_ref().replace("\r\n", "\n");
+    let org_contents = org_contents.as_str();
+    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
+    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
+    let rust_parsed = parse(org_contents)?;
+    let org_sexp = emacs_parse_anonymous_org_document(org_contents)?;
+    let (_remaining, parsed_sexp) = sexp(org_sexp.as_str()).map_err(|e| e.to_string())?;
+
+    println!("{}\n\n\n", org_contents);
+    println!("{}", org_sexp);
+    println!("{:#?}", rust_parsed);
+
+    // We do the diffing after printing out both parsed forms in case the diffing panics
+    let diff_result = compare_document(&parsed_sexp, &rust_parsed)?;
+    diff_result.print(org_contents)?;
+
+    if diff_result.is_bad() {
+        Err("Diff results do not match.")?;
+    }
+
+    Ok(())
+}
+
+pub fn run_compare_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn std::error::Error>> {
+    let org_path = org_path.as_ref();
+    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
+    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
+    let parent_directory = org_path
+        .parent()
+        .ok_or("Should be contained inside a directory.")?;
+    let org_contents = std::fs::read_to_string(org_path)?;
+    // TODO: This is a work-around to pretend that dos line endings do not exist. It would be better to handle the difference in line endings.
+    let org_contents = org_contents.replace("\r\n", "\n");
+    let org_contents = org_contents.as_str();
+    let file_access_interface = LocalFileAccessInterface {
+        working_directory: Some(parent_directory.to_path_buf()),
+    };
+    let global_settings = {
+        let mut global_settings = GlobalSettings::default();
+        global_settings.file_access = &file_access_interface;
+        global_settings
+    };
+    let rust_parsed = parse_with_settings(org_contents, &global_settings)?;
+    let org_sexp = emacs_parse_file_org_document(org_path)?;
+    let (_remaining, parsed_sexp) = sexp(org_sexp.as_str()).map_err(|e| e.to_string())?;
+
+    println!("{}\n\n\n", org_contents);
+    println!("{}", org_sexp);
+    println!("{:#?}", rust_parsed);
+
+    // We do the diffing after printing out both parsed forms in case the diffing panics
+    let diff_result = compare_document(&parsed_sexp, &rust_parsed)?;
+    diff_result.print(org_contents)?;
+
+    if diff_result.is_bad() {
+        Err("Diff results do not match.")?;
+    }
+
+    Ok(())
+}

src/compare/elisp_fact.rs

@@ -0,0 +1,474 @@
+use std::borrow::Cow;
+
+use crate::types::AngleLink;
+use crate::types::BabelCall;
+use crate::types::Bold;
+use crate::types::Citation;
+use crate::types::CitationReference;
+use crate::types::Clock;
+use crate::types::Code;
+use crate::types::Comment;
+use crate::types::CommentBlock;
+use crate::types::DiarySexp;
+use crate::types::Document;
+use crate::types::Drawer;
+use crate::types::DynamicBlock;
+use crate::types::Element;
+use crate::types::Entity;
+use crate::types::ExampleBlock;
+use crate::types::ExportBlock;
+use crate::types::ExportSnippet;
+use crate::types::FixedWidthArea;
+use crate::types::FootnoteDefinition;
+use crate::types::FootnoteReference;
+use crate::types::GreaterBlock;
+use crate::types::Heading;
+use crate::types::HorizontalRule;
+use crate::types::InlineBabelCall;
+use crate::types::InlineSourceBlock;
+use crate::types::Italic;
+use crate::types::Keyword;
+use crate::types::LatexEnvironment;
+use crate::types::LatexFragment;
+use crate::types::LineBreak;
+use crate::types::NodeProperty;
+use crate::types::Object;
+use crate::types::OrgMacro;
+use crate::types::Paragraph;
+use crate::types::PlainLink;
+use crate::types::PlainList;
+use crate::types::PlainListItem;
+use crate::types::PlainText;
+use crate::types::Planning;
+use crate::types::PropertyDrawer;
+use crate::types::RadioLink;
+use crate::types::RadioTarget;
+use crate::types::RegularLink;
+use crate::types::Section;
+use crate::types::SrcBlock;
+use crate::types::StatisticsCookie;
+use crate::types::StrikeThrough;
+use crate::types::Subscript;
+use crate::types::Superscript;
+use crate::types::Table;
+use crate::types::TableCell;
+use crate::types::TableRow;
+use crate::types::Target;
+use crate::types::Timestamp;
+use crate::types::Underline;
+use crate::types::Verbatim;
+use crate::types::VerseBlock;
+
+pub(crate) trait ElispFact<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str>;
+}
+
+pub(crate) trait GetElispFact<'s> {
+    fn get_elisp_fact(&'s self) -> &'s dyn ElispFact<'s>;
+}
+
+impl<'s, I: ElispFact<'s>> GetElispFact<'s> for I {
+    fn get_elisp_fact(&'s self) -> &'s dyn ElispFact<'s> {
+        self
+    }
+}
+
+impl<'s> GetElispFact<'s> for Element<'s> {
+    fn get_elisp_fact(&'s self) -> &'s dyn ElispFact<'s> {
+        match self {
+            Element::Paragraph(inner) => inner,
+            Element::PlainList(inner) => inner,
+            Element::GreaterBlock(inner) => inner,
+            Element::DynamicBlock(inner) => inner,
+            Element::FootnoteDefinition(inner) => inner,
+            Element::Comment(inner) => inner,
+            Element::Drawer(inner) => inner,
+            Element::PropertyDrawer(inner) => inner,
+            Element::Table(inner) => inner,
+            Element::VerseBlock(inner) => inner,
+            Element::CommentBlock(inner) => inner,
+            Element::ExampleBlock(inner) => inner,
+            Element::ExportBlock(inner) => inner,
+            Element::SrcBlock(inner) => inner,
+            Element::Clock(inner) => inner,
+            Element::DiarySexp(inner) => inner,
+            Element::Planning(inner) => inner,
+            Element::FixedWidthArea(inner) => inner,
+            Element::HorizontalRule(inner) => inner,
+            Element::Keyword(inner) => inner,
+            Element::BabelCall(inner) => inner,
+            Element::LatexEnvironment(inner) => inner,
+        }
+    }
+}
+
+impl<'s> GetElispFact<'s> for Object<'s> {
+    fn get_elisp_fact(&'s self) -> &'s dyn ElispFact<'s> {
+        match self {
+            Object::Bold(inner) => inner,
+            Object::Italic(inner) => inner,
+            Object::Underline(inner) => inner,
+            Object::StrikeThrough(inner) => inner,
+            Object::Code(inner) => inner,
+            Object::Verbatim(inner) => inner,
+            Object::PlainText(inner) => inner,
+            Object::RegularLink(inner) => inner,
+            Object::RadioLink(inner) => inner,
+            Object::RadioTarget(inner) => inner,
+            Object::PlainLink(inner) => inner,
+            Object::AngleLink(inner) => inner,
+            Object::OrgMacro(inner) => inner,
+            Object::Entity(inner) => inner,
+            Object::LatexFragment(inner) => inner,
+            Object::ExportSnippet(inner) => inner,
+            Object::FootnoteReference(inner) => inner,
+            Object::Citation(inner) => inner,
+            Object::CitationReference(inner) => inner,
+            Object::InlineBabelCall(inner) => inner,
+            Object::InlineSourceBlock(inner) => inner,
+            Object::LineBreak(inner) => inner,
+            Object::Target(inner) => inner,
+            Object::StatisticsCookie(inner) => inner,
+            Object::Subscript(inner) => inner,
+            Object::Superscript(inner) => inner,
+            Object::Timestamp(inner) => inner,
+        }
+    }
+}
+
+impl<'s> ElispFact<'s> for Document<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "org-data".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Section<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "section".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Heading<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "headline".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainList<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "plain-list".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainListItem<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "item".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for GreaterBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        match self.name.to_lowercase().as_str() {
+            "center" => "center-block".into(),
+            "quote" => "quote-block".into(),
+            _ => "special-block".into(),
+        }
+    }
+}
+
+impl<'s> ElispFact<'s> for DynamicBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "dynamic-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for FootnoteDefinition<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "footnote-definition".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Drawer<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "drawer".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PropertyDrawer<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "property-drawer".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for NodeProperty<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "node-property".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Table<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "table".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for TableRow<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "table-row".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Paragraph<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "paragraph".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for TableCell<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "table-cell".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Comment<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "comment".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for VerseBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "verse-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for CommentBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "comment-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for ExampleBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "example-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for ExportBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "export-block".into()
+    }
+}
+impl<'s> ElispFact<'s> for SrcBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "src-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Clock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "clock".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for DiarySexp<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "diary-sexp".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Planning<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "planning".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for FixedWidthArea<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "fixed-width".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for HorizontalRule<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "horizontal-rule".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Keyword<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "keyword".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for BabelCall<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "babel-call".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for LatexEnvironment<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "latex-environment".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Bold<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "bold".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Italic<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "italic".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Underline<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "underline".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for StrikeThrough<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "strike-through".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Code<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "code".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Verbatim<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "verbatim".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for RegularLink<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for RadioLink<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for RadioTarget<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "radio-target".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainLink<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for AngleLink<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "link".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for OrgMacro<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "macro".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Entity<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "entity".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for LatexFragment<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "latex-fragment".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for ExportSnippet<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "export-snippet".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for FootnoteReference<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "footnote-reference".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Citation<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "citation".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for CitationReference<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "citation-reference".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for InlineBabelCall<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "inline-babel-call".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for InlineSourceBlock<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "inline-src-block".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for LineBreak<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "line-break".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Target<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "target".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for StatisticsCookie<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "statistics-cookie".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Subscript<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "subscript".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Superscript<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "superscript".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for Timestamp<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        "timestamp".into()
+    }
+}
+
+impl<'s> ElispFact<'s> for PlainText<'s> {
+    fn get_elisp_name(&'s self) -> Cow<'s, str> {
+        // plain text from upstream emacs does not actually have a name but this is included here to make rendering the status diff easier.
+        "plain-text".into()
+    }
+}

src/compare/mod.rs

@@ -1,8 +1,8 @@
+mod compare;
 mod diff;
+mod elisp_fact;
 mod parse;
+mod sexp;
 mod util;
-pub use diff::compare_document;
-pub use parse::emacs_parse_anonymous_org_document;
-pub use parse::emacs_parse_file_org_document;
-pub use parse::get_emacs_version;
-pub use parse::get_org_mode_version;
+pub use compare::run_anonymous_compare;
+pub use compare::run_compare_on_file;

src/compare/parse.rs

@@ -11,6 +11,8 @@ where
     let elisp_script = format!(
         r#"(progn
      (erase-buffer)
+     (require 'org)
+     (defun org-table-align () t)
      (insert "{escaped_file_contents}")
      (org-mode)
      (message "%s" (pp-to-string (org-element-parse-buffer)))
@@ -42,6 +44,8 @@ where
     ))?;
     let elisp_script = format!(
         r#"(progn
+     (require 'org)
+     (defun org-table-align () t)
      (org-mode)
      (message "%s" (pp-to-string (org-element-parse-buffer)))
 )"#

src/compare/sexp.rs

@@ -16,9 +16,6 @@ use nom::sequence::delimited;
 use nom::sequence::preceded;
 use nom::sequence::tuple;
 
-use super::org_source::convert_error;
-use super::org_source::OrgSource;
-use super::util::get_consumed;
 use crate::error::Res;
 
 #[derive(Debug)]
@@ -31,8 +28,8 @@ pub enum Token<'s> {
 
 #[derive(Debug)]
 pub struct TextWithProperties<'s> {
-    pub text: &'s str,
-    pub properties: Vec<Token<'s>>,
+    pub(crate) text: &'s str,
+    pub(crate) properties: Vec<Token<'s>>,
 }
 
 enum ParseState {
@@ -41,35 +38,39 @@ enum ParseState {
 }
 
 impl<'s> Token<'s> {
-    pub fn as_vector<'p>(&'p self) -> Result<&'p Vec<Token<'s>>, Box<dyn std::error::Error>> {
+    pub(crate) fn as_vector<'p>(
+        &'p self,
+    ) -> Result<&'p Vec<Token<'s>>, Box<dyn std::error::Error>> {
         Ok(match self {
             Token::Vector(children) => Ok(children),
             _ => Err(format!("wrong token type, expected vector: {:?}", self)),
         }?)
     }
 
-    pub fn as_list<'p>(&'p self) -> Result<&'p Vec<Token<'s>>, Box<dyn std::error::Error>> {
+    pub(crate) fn as_list<'p>(&'p self) -> Result<&'p Vec<Token<'s>>, Box<dyn std::error::Error>> {
         Ok(match self {
             Token::List(children) => Ok(children),
             _ => Err(format!("wrong token type, expected list: {:?}", self)),
         }?)
     }
 
-    pub fn as_atom<'p>(&'p self) -> Result<&'s str, Box<dyn std::error::Error>> {
+    pub(crate) fn as_atom<'p>(&'p self) -> Result<&'s str, Box<dyn std::error::Error>> {
         Ok(match self {
             Token::Atom(body) => Ok(*body),
             _ => Err(format!("wrong token type, expected atom: {:?}", self)),
         }?)
     }
 
-    pub fn as_text<'p>(&'p self) -> Result<&'p TextWithProperties<'s>, Box<dyn std::error::Error>> {
+    pub(crate) fn as_text<'p>(
+        &'p self,
+    ) -> Result<&'p TextWithProperties<'s>, Box<dyn std::error::Error>> {
         Ok(match self {
             Token::TextWithProperties(body) => Ok(body),
             _ => Err(format!("wrong token type, expected text: {:?}", self)),
         }?)
     }
 
-    pub fn as_map<'p>(
+    pub(crate) fn as_map<'p>(
         &'p self,
     ) -> Result<HashMap<&'s str, &'p Token<'s>>, Box<dyn std::error::Error>> {
         let mut hashmap = HashMap::new();
@@ -95,7 +96,26 @@ impl<'s> Token<'s> {
     }
 }
 
-pub fn unquote(text: &str) -> Result<String, Box<dyn std::error::Error>> {
+/// Check if the child string slice is a slice of the parent string slice.
+fn is_slice_of(parent: &str, child: &str) -> bool {
+    let parent_start = parent.as_ptr() as usize;
+    let parent_end = parent_start + parent.len();
+    let child_start = child.as_ptr() as usize;
+    let child_end = child_start + child.len();
+    child_start >= parent_start && child_end <= parent_end
+}
+
+/// Get a slice of the string that was consumed in a parser using the original input to the parser and the remaining input after the parser.
+fn get_consumed<'s>(input: &'s str, remaining: &'s str) -> &'s str {
+    debug_assert!(is_slice_of(input, remaining));
+    let source = {
+        let offset = remaining.as_ptr() as usize - input.as_ptr() as usize;
+        &input[..offset]
+    };
+    source.into()
+}
+
+pub(crate) fn unquote(text: &str) -> Result<String, Box<dyn std::error::Error>> {
     let mut out = String::with_capacity(text.len());
     if !text.starts_with(r#"""#) {
         return Err("Quoted text does not start with quote.".into());
@@ -132,29 +152,20 @@ pub fn unquote(text: &str) -> Result<String, Box<dyn std::error::Error>> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn sexp_with_padding<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
+pub fn sexp<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, _) = multispace0(input)?;
-    let remaining = OrgSource::new(remaining);
-    let (remaining, tkn) = token(remaining)
-        .map(|(rem, out)| (Into::<&str>::into(rem), out))
-        .map_err(convert_error)?;
+    let (remaining, tkn) = token(remaining).map(|(rem, out)| (Into::<&str>::into(rem), out))?;
     let (remaining, _) = multispace0(remaining)?;
     Ok((remaining, tkn))
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn sexp<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
-    let (remaining, tkn) = token(input)?;
-    Ok((remaining, tkn))
-}
-
-#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn token<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn token<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     alt((list, vector, atom))(input)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn list<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn list<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, _) = tag("(")(input)?;
     let (remaining, children) = delimited(
         multispace0,
@@ -166,7 +177,7 @@ fn list<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn vector<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn vector<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, _) = tag("[")(input)?;
     let (remaining, children) = delimited(
         multispace0,
@@ -178,7 +189,7 @@ fn vector<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn atom<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn atom<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     not(peek(one_of(")]")))(input)?;
     alt((
         text_with_properties,
@@ -189,7 +200,7 @@ fn atom<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn unquoted_atom<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn unquoted_atom<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, body) = take_till1(|c| match c {
         ' ' | '\t' | '\r' | '\n' | ')' | ']' => true,
         _ => false,
@@ -198,7 +209,7 @@ fn unquoted_atom<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn quoted_atom<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn quoted_atom<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, _) = tag(r#"""#)(input)?;
     let (remaining, _) = escaped(
         take_till1(|c| match c {
@@ -214,7 +225,7 @@ fn quoted_atom<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn hash_notation<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn hash_notation<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, _) = tag("#<")(input)?;
     let (remaining, _body) = take_till1(|c| match c {
         '>' => true,
@@ -225,7 +236,7 @@ fn hash_notation<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
     Ok((remaining, Token::Atom(source.into())))
 }
 
-fn text_with_properties<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, Token<'s>> {
+fn text_with_properties<'s>(input: &'s str) -> Res<&'s str, Token<'s>> {
     let (remaining, _) = tag("#(")(input)?;
     let (remaining, (text, props)) = delimited(
         multispace0,
@@ -255,7 +266,7 @@ mod tests {
     #[test]
     fn simple() {
         let input = "  (foo bar baz )  ";
-        let (remaining, parsed) = sexp_with_padding(input).expect("Parse the input");
+        let (remaining, parsed) = sexp(input).expect("Parse the input");
         assert_eq!(remaining, "");
         assert!(match parsed {
             Token::Atom(_) => false,
@@ -268,7 +279,7 @@ mod tests {
     #[test]
     fn quoted() {
         let input = r#"  ("foo" bar baz )  "#;
-        let (remaining, parsed) = sexp_with_padding(input).expect("Parse the input");
+        let (remaining, parsed) = sexp(input).expect("Parse the input");
         assert_eq!(remaining, "");
         assert!(match parsed {
             Token::Atom(_) => false,
@@ -292,7 +303,7 @@ mod tests {
     #[test]
     fn quoted_containing_paren() {
         let input = r#"  (foo "b(a)r" baz )  "#;
-        let (remaining, parsed) = sexp_with_padding(input).expect("Parse the input");
+        let (remaining, parsed) = sexp(input).expect("Parse the input");
         assert_eq!(remaining, "");
         assert!(match parsed {
             Token::List(_) => true,
@@ -328,7 +339,7 @@ mod tests {
     #[test]
     fn string_containing_escaped_characters() {
         let input = r#"  (foo "\\( x=2 \\)" bar)  "#;
-        let (remaining, parsed) = sexp_with_padding(input).expect("Parse the input");
+        let (remaining, parsed) = sexp(input).expect("Parse the input");
         assert_eq!(remaining, "");
         assert!(match parsed {
             Token::Atom(_) => false,

src/compare/util.rs

@@ -1,5 +1,7 @@
-use crate::parser::sexp::Token;
-use crate::types::Source;
+use super::elisp_fact::GetElispFact;
+use super::sexp::Token;
+use crate::types::GetStandardProperties;
+use crate::types::StandardProperties;
 
 /// Check if the child string slice is a slice of the parent string slice.
 fn is_slice_of(parent: &str, child: &str) -> bool {
@@ -10,18 +12,38 @@ fn is_slice_of(parent: &str, child: &str) -> bool {
     child_start >= parent_start && child_end <= parent_end
 }
 
-/// Get the offset into source that the rust object exists at.
+/// Get the byte offset into source that the rust object exists at.
 ///
 /// These offsets are zero-based unlike the elisp ones.
-fn get_offsets<'s, S: Source<'s>>(source: &'s str, rust_object: &'s S) -> (usize, usize) {
-    let rust_object_source = rust_object.get_source();
-    assert!(is_slice_of(source, rust_object_source));
-    let offset = rust_object_source.as_ptr() as usize - source.as_ptr() as usize;
+fn get_rust_byte_offsets<'s, S: StandardProperties<'s> + ?Sized>(
+    original_document: &'s str,
+    rust_ast_node: &'s S,
+) -> (usize, usize) {
+    let rust_object_source = rust_ast_node.get_source();
+    debug_assert!(is_slice_of(original_document, rust_object_source));
+    let offset = rust_object_source.as_ptr() as usize - original_document.as_ptr() as usize;
     let end = offset + rust_object_source.len();
     (offset, end)
 }
 
-pub fn assert_name<'s>(emacs: &'s Token<'s>, name: &str) -> Result<(), Box<dyn std::error::Error>> {
+pub(crate) fn compare_standard_properties<
+    's,
+    S: GetStandardProperties<'s> + GetElispFact<'s> + ?Sized,
+>(
+    original_document: &'s str,
+    emacs: &'s Token<'s>,
+    rust: &'s S,
+) -> Result<(), Box<dyn std::error::Error>> {
+    assert_name(emacs, rust.get_elisp_fact().get_elisp_name())?;
+    assert_bounds(original_document, emacs, rust.get_standard_properties())?;
+    Ok(())
+}
+
+pub(crate) fn assert_name<'s, S: AsRef<str>>(
+    emacs: &'s Token<'s>,
+    name: S,
+) -> Result<(), Box<dyn std::error::Error>> {
+    let name = name.as_ref();
     let children = emacs.as_list()?;
     let first_child = children
         .first()
@@ -29,7 +51,7 @@ pub fn assert_name<'s>(emacs: &'s Token<'s>, name: &str) -> Result<(), Box<dyn s
         .as_atom()?;
     if first_child != name {
         Err(format!(
-            "Expected a {expected} cell, but found a {found} cell.",
+            "AST node name mismatch. Expected a (rust) {expected} cell, but found a (emacs) {found} cell.",
             expected = name,
             found = first_child
         ))?;
@@ -37,30 +59,33 @@ pub fn assert_name<'s>(emacs: &'s Token<'s>, name: &str) -> Result<(), Box<dyn s
     Ok(())
 }
 
-pub fn assert_bounds<'s, S: Source<'s>>(
-    source: &'s str,
+/// Assert that the character ranges defined by upstream org-mode's :standard-properties match the slices in Organic's StandardProperties.
+///
+/// This does **not** handle plain text because plain text is a special case.
+pub(crate) fn assert_bounds<'s, S: StandardProperties<'s> + ?Sized>(
+    original_document: &'s str,
     emacs: &'s Token<'s>,
     rust: &'s S,
 ) -> Result<(), Box<dyn std::error::Error>> {
-    let standard_properties = get_standard_properties(emacs)?;
+    let standard_properties = get_emacs_standard_properties(emacs)?; // 1-based
     let (begin, end) = (
         standard_properties
             .begin
             .ok_or("Token should have a begin.")?,
         standard_properties.end.ok_or("Token should have an end.")?,
     );
-    let (rust_begin, rust_end) = get_offsets(source, rust);
-    let rust_begin_char_offset = (&source[..rust_begin]).chars().count();
+    let (rust_begin, rust_end) = get_rust_byte_offsets(original_document, rust); // 0-based
+    let rust_begin_char_offset = (&original_document[..rust_begin]).chars().count() + 1; // 1-based
     let rust_end_char_offset =
-        rust_begin_char_offset + (&source[rust_begin..rust_end]).chars().count();
-    if (rust_begin_char_offset + 1) != begin || (rust_end_char_offset + 1) != end {
-        Err(format!("Rust bounds (in chars) ({rust_begin}, {rust_end}) do not match emacs bounds ({emacs_begin}, {emacs_end})", rust_begin = rust_begin_char_offset + 1, rust_end = rust_end_char_offset + 1, emacs_begin=begin, emacs_end=end))?;
+        rust_begin_char_offset + (&original_document[rust_begin..rust_end]).chars().count(); // 1-based
+    if rust_begin_char_offset != begin || rust_end_char_offset != end {
+        Err(format!("Rust bounds (in chars) ({rust_begin}, {rust_end}) do not match emacs bounds ({emacs_begin}, {emacs_end})", rust_begin = rust_begin_char_offset, rust_end = rust_end_char_offset, emacs_begin=begin, emacs_end=end))?;
     }
 
     Ok(())
 }
 
-struct StandardProperties {
+struct EmacsStandardProperties {
     begin: Option<usize>,
     #[allow(dead_code)]
     post_affiliated: Option<usize>,
@@ -73,9 +98,9 @@ struct StandardProperties {
     post_blank: Option<usize>,
 }
 
-fn get_standard_properties<'s>(
+fn get_emacs_standard_properties<'s>(
     emacs: &'s Token<'s>,
-) -> Result<StandardProperties, Box<dyn std::error::Error>> {
+) -> Result<EmacsStandardProperties, Box<dyn std::error::Error>> {
     let children = emacs.as_list()?;
     let attributes_child = children
         .iter()
@@ -94,7 +119,7 @@ fn get_standard_properties<'s>(
         let contents_end = maybe_token_to_usize(std_props.next())?;
         let end = maybe_token_to_usize(std_props.next())?;
         let post_blank = maybe_token_to_usize(std_props.next())?;
-        StandardProperties {
+        EmacsStandardProperties {
             begin,
             post_affiliated,
             contents_begin,
@@ -113,7 +138,7 @@ fn get_standard_properties<'s>(
             maybe_token_to_usize(attributes_map.get(":post-blank").map(|token| *token))?;
         let post_affiliated =
             maybe_token_to_usize(attributes_map.get(":post-affiliated").map(|token| *token))?;
-        StandardProperties {
+        EmacsStandardProperties {
             begin,
             post_affiliated,
             contents_begin,
@@ -146,7 +171,7 @@ fn maybe_token_to_usize(
 /// Returns Ok(None) if value is nil.
 ///
 /// Returns error if the attribute is not specified on the token at all.
-pub fn get_property<'s, 'x>(
+pub(crate) fn get_property<'s, 'x>(
     emacs: &'s Token<'s>,
     key: &'x str,
 ) -> Result<Option<&'s Token<'s>>, Box<dyn std::error::Error>> {

src/context/context.rs

@@ -14,7 +14,7 @@ use crate::error::Res;
 use crate::parser::OrgSource;
 
 #[derive(Debug)]
-pub enum ContextElement<'r, 's> {
+pub(crate) enum ContextElement<'r, 's> {
     /// Stores a parser that indicates that children should exit upon matching an exit matcher.
     ExitMatcherNode(ExitMatcherNode<'r>),
 
@@ -28,13 +28,14 @@ pub enum ContextElement<'r, 's> {
     ConsumeTrailingWhitespace(bool),
 
     /// This is just here to use the 's lifetime until I'm sure we can eliminate it from ContextElement.
+    #[allow(dead_code)]
     Placeholder(PhantomData<&'s str>),
 }
 
-pub struct ExitMatcherNode<'r> {
+pub(crate) struct ExitMatcherNode<'r> {
     // TODO: Should this be "&'r DynContextMatcher<'c>" ?
-    pub exit_matcher: &'r DynContextMatcher<'r>,
-    pub class: ExitClass,
+    pub(crate) exit_matcher: &'r DynContextMatcher<'r>,
+    pub(crate) class: ExitClass,
 }
 
 impl<'r> std::fmt::Debug for ExitMatcherNode<'r> {
@@ -46,13 +47,13 @@ impl<'r> std::fmt::Debug for ExitMatcherNode<'r> {
 }
 
 #[derive(Debug)]
-pub struct Context<'g, 'r, 's> {
+pub(crate) struct Context<'g, 'r, 's> {
     global_settings: &'g GlobalSettings<'g, 's>,
     tree: List<'r, &'r ContextElement<'r, 's>>,
 }
 
 impl<'g, 'r, 's> Context<'g, 'r, 's> {
-    pub fn new(
+    pub(crate) fn new(
         global_settings: &'g GlobalSettings<'g, 's>,
         tree: List<'r, &'r ContextElement<'r, 's>>,
     ) -> Self {
@@ -62,38 +63,38 @@ impl<'g, 'r, 's> Context<'g, 'r, 's> {
         }
     }
 
-    pub fn with_additional_node(&'r self, new_element: &'r ContextElement<'r, 's>) -> Self {
+    pub(crate) fn with_additional_node(&'r self, new_element: &'r ContextElement<'r, 's>) -> Self {
         let new_tree = self.tree.push(new_element);
         Self::new(self.global_settings, new_tree)
     }
 
-    pub fn iter(&'r self) -> super::list::Iter<'r, &'r ContextElement<'r, 's>> {
+    pub(crate) fn iter(&'r self) -> super::list::Iter<'r, &'r ContextElement<'r, 's>> {
         self.tree.iter()
     }
 
-    pub fn iter_context(&'r self) -> Iter<'g, 'r, 's> {
+    fn iter_context(&'r self) -> Iter<'g, 'r, 's> {
         Iter {
             next: self.tree.iter_list(),
             global_settings: self.global_settings,
         }
     }
 
-    pub fn get_parent(&'r self) -> Option<Self> {
+    pub(crate) fn get_parent(&'r self) -> Option<Self> {
         self.tree.get_parent().map(|parent_tree| Self {
             global_settings: self.global_settings,
             tree: parent_tree.clone(),
         })
     }
 
-    pub fn get_data(&self) -> &ContextElement<'r, 's> {
+    fn get_data(&self) -> &ContextElement<'r, 's> {
         self.tree.get_data()
     }
 
-    pub fn get_global_settings(&self) -> &'g GlobalSettings<'g, 's> {
+    pub(crate) fn get_global_settings(&self) -> &'g GlobalSettings<'g, 's> {
         self.global_settings
     }
 
-    pub fn with_global_settings<'gg>(
+    pub(crate) fn with_global_settings<'gg>(
         &self,
         new_settings: &'gg GlobalSettings<'gg, 's>,
     ) -> Context<'gg, 'r, 's> {
@@ -104,11 +105,11 @@ impl<'g, 'r, 's> Context<'g, 'r, 's> {
     }
 
     #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-    pub fn check_exit_matcher(
+    pub(crate) fn check_exit_matcher(
         &'r self,
         i: OrgSource<'s>,
     ) -> IResult<OrgSource<'s>, OrgSource<'s>, CustomError<OrgSource<'s>>> {
-        let mut current_class_filter = ExitClass::Delta;
+        let mut current_class_filter = ExitClass::Gamma;
         for current_node in self.iter_context() {
             let context_element = current_node.get_data();
             match context_element {
@@ -133,7 +134,7 @@ impl<'g, 'r, 's> Context<'g, 'r, 's> {
     /// Indicates if elements should consume the whitespace after them.
     ///
     /// Defaults to true.
-    pub fn should_consume_trailing_whitespace(&self) -> bool {
+    pub(crate) fn should_consume_trailing_whitespace(&self) -> bool {
         self._should_consume_trailing_whitespace().unwrap_or(true)
     }
 
@@ -158,7 +159,7 @@ fn document_end<'b, 'g, 'r, 's>(
     eof(input)
 }
 
-pub struct Iter<'g, 'r, 's> {
+struct Iter<'g, 'r, 's> {
     global_settings: &'g GlobalSettings<'g, 's>,
     next: super::list::IterList<'r, &'r ContextElement<'r, 's>>,
 }
@@ -175,7 +176,7 @@ impl<'g, 'r, 's> Iterator for Iter<'g, 'r, 's> {
 }
 
 impl<'r, 's> ContextElement<'r, 's> {
-    pub fn document_context() -> Self {
+    pub(crate) fn document_context() -> Self {
         Self::ExitMatcherNode(ExitMatcherNode {
             exit_matcher: &document_end,
             class: ExitClass::Document,

src/context/exiting.rs

@@ -1,10 +1,9 @@
 #[derive(Debug, Copy, Clone)]
-pub enum ExitClass {
+pub(crate) enum ExitClass {
     Document = 1,
     Alpha = 2,
     Beta = 3,
     Gamma = 4,
-    Delta = 5,
 }
 
 impl std::fmt::Display for ExitClass {

src/context/global_settings.rs

@@ -2,6 +2,7 @@ use std::collections::BTreeSet;
 
 use super::FileAccessInterface;
 use super::LocalFileAccessInterface;
+use crate::types::IndentationLevel;
 use crate::types::Object;
 
 // TODO: Ultimately, I think we'll need most of this: https://orgmode.org/manual/In_002dbuffer-Settings.html
@@ -12,10 +13,24 @@ pub struct GlobalSettings<'g, 's> {
     pub file_access: &'g dyn FileAccessInterface,
     pub in_progress_todo_keywords: BTreeSet<String>,
     pub complete_todo_keywords: BTreeSet<String>,
+    /// Set to true to allow for plain lists using single letters as the bullet in the same way that numbers are used.
+    ///
+    /// Corresponds to the org-list-allow-alphabetical elisp variable.
+    pub org_list_allow_alphabetical: bool,
+
+    /// How many spaces a tab should be equal to.
+    ///
+    /// Corresponds to the tab-width elisp variable.
+    pub tab_width: IndentationLevel,
+
+    /// Whether to only allow odd headline levels.
+    ///
+    /// Corresponds to org-odd-levels-only elisp variable.
+    pub odd_levels_only: HeadlineLevelFilter,
 }
 
 impl<'g, 's> GlobalSettings<'g, 's> {
-    pub fn new() -> GlobalSettings<'g, 's> {
+    fn new() -> GlobalSettings<'g, 's> {
         GlobalSettings {
             radio_targets: Vec::new(),
             file_access: &LocalFileAccessInterface {
@@ -23,6 +38,9 @@ impl<'g, 's> GlobalSettings<'g, 's> {
             },
             in_progress_todo_keywords: BTreeSet::new(),
             complete_todo_keywords: BTreeSet::new(),
+            org_list_allow_alphabetical: false,
+            tab_width: 8,
+            odd_levels_only: HeadlineLevelFilter::OddEven,
         }
     }
 }
@@ -32,3 +50,9 @@ impl<'g, 's> Default for GlobalSettings<'g, 's> {
         GlobalSettings::new()
     }
 }
+
+#[derive(Debug, Clone)]
+pub enum HeadlineLevelFilter {
+    Odd,
+    OddEven,
+}

src/context/list.rs

@@ -1,38 +1,39 @@
 use std::fmt::Debug;
 
 #[derive(Debug, Clone)]
-pub struct List<'parent, T> {
+pub(crate) struct List<'parent, T> {
     data: T,
     parent: Link<'parent, T>,
 }
 
+// TODO: Should I be defining a lifetime for T in the generics here? Ref: https://quinedot.github.io/rust-learning/dyn-elision-advanced.html#iteraction-with-type-aliases
 type Link<'parent, T> = Option<&'parent List<'parent, T>>;
 
 impl<'parent, T> List<'parent, T> {
-    pub fn new(first_item: T) -> Self {
+    pub(crate) fn new(first_item: T) -> Self {
         Self {
             data: first_item,
             parent: None,
         }
     }
 
-    pub fn get_data(&self) -> &T {
+    pub(crate) fn get_data(&self) -> &T {
         &self.data
     }
 
-    pub fn get_parent(&'parent self) -> Link<'parent, T> {
+    pub(crate) fn get_parent(&'parent self) -> Link<'parent, T> {
         self.parent
     }
 
-    pub fn iter(&self) -> Iter<'_, T> {
+    pub(crate) fn iter(&self) -> Iter<'_, T> {
         Iter { next: Some(self) }
     }
 
-    pub fn iter_list(&self) -> IterList<'_, T> {
+    pub(crate) fn iter_list(&self) -> IterList<'_, T> {
         IterList { next: Some(self) }
     }
 
-    pub fn push(&'parent self, item: T) -> Self {
+    pub(crate) fn push(&'parent self, item: T) -> Self {
         Self {
             data: item,
             parent: Some(self),
@@ -40,7 +41,7 @@ impl<'parent, T> List<'parent, T> {
     }
 }
 
-pub struct Iter<'a, T> {
+pub(crate) struct Iter<'a, T> {
     next: Link<'a, T>,
 }
 
@@ -54,7 +55,7 @@ impl<'a, T> Iterator for Iter<'a, T> {
     }
 }
 
-pub struct IterList<'a, T> {
+pub(crate) struct IterList<'a, T> {
     next: Link<'a, T>,
 }
 

src/context/mod.rs

@@ -8,22 +8,23 @@ mod global_settings;
 mod list;
 mod parser_with_context;
 
-pub type RefContext<'b, 'g, 'r, 's> = &'b Context<'g, 'r, 's>;
-pub trait ContextMatcher = for<'b, 'g, 'r, 's> Fn(
+pub(crate) type RefContext<'b, 'g, 'r, 's> = &'b Context<'g, 'r, 's>;
+pub(crate) trait ContextMatcher = for<'b, 'g, 'r, 's> Fn(
     RefContext<'b, 'g, 'r, 's>,
     OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>>;
-pub type DynContextMatcher<'c> = dyn ContextMatcher + 'c;
-pub trait Matcher = for<'s> Fn(OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>>;
+type DynContextMatcher<'c> = dyn ContextMatcher + 'c;
+pub(crate) trait Matcher = for<'s> Fn(OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>>;
 #[allow(dead_code)]
-pub type DynMatcher<'c> = dyn Matcher + 'c;
+type DynMatcher<'c> = dyn Matcher + 'c;
 
-pub use context::Context;
-pub use context::ContextElement;
-pub use context::ExitMatcherNode;
-pub use exiting::ExitClass;
+pub(crate) use context::Context;
+pub(crate) use context::ContextElement;
+pub(crate) use context::ExitMatcherNode;
+pub(crate) use exiting::ExitClass;
 pub use file_access_interface::FileAccessInterface;
 pub use file_access_interface::LocalFileAccessInterface;
 pub use global_settings::GlobalSettings;
-pub use list::List;
+pub use global_settings::HeadlineLevelFilter;
+pub(crate) use list::List;
 pub(crate) use parser_with_context::parser_with_context;

src/error/error.rs

@@ -2,7 +2,7 @@ use nom::error::ErrorKind;
 use nom::error::ParseError;
 use nom::IResult;
 
-pub type Res<T, U> = IResult<T, U, CustomError<T>>;
+pub(crate) type Res<T, U> = IResult<T, U, CustomError<T>>;
 
 // TODO: MyError probably shouldn't be based on the same type as the input type since it's used exclusively with static strings right now.
 #[derive(Debug)]
@@ -13,7 +13,7 @@ pub enum CustomError<I> {
 }
 
 #[derive(Debug)]
-pub struct MyError<I>(pub I);
+pub struct MyError<I>(pub(crate) I);
 
 impl<I> ParseError<I> for CustomError<I> {
     fn from_error_kind(input: I, kind: ErrorKind) -> Self {

src/error/mod.rs

@@ -1,4 +1,4 @@
 mod error;
-pub use error::CustomError;
-pub use error::MyError;
-pub use error::Res;
+pub(crate) use error::CustomError;
+pub(crate) use error::MyError;
+pub(crate) use error::Res;

src/init_tracing.rs

@@ -10,7 +10,7 @@ const SERVICE_NAME: &'static str = "organic";
 // Despite the obvious verbosity that fully-qualifying everything causes, in these functions I am fully-qualifying everything relating to tracing. This is because the tracing feature involves multiple libraries working together and so I think it is beneficial to see which libraries contribute which bits.
 
 #[cfg(feature = "tracing")]
-pub fn init_telemetry() -> Result<(), Box<dyn std::error::Error>> {
+pub(crate) fn init_telemetry() -> Result<(), Box<dyn std::error::Error>> {
     // by default it will hit http://localhost:4317 with a gRPC payload
     // TODO: I think the endpoint can be controlled by the OTEL_EXPORTER_OTLP_TRACES_ENDPOINT env variable instead of hard-coded into this code base. Regardless, I am the only developer right now so I am not too concerned.
     let exporter = opentelemetry_otlp::new_exporter()
@@ -55,17 +55,17 @@ pub fn init_telemetry() -> Result<(), Box<dyn std::error::Error>> {
 }
 
 #[cfg(feature = "tracing")]
-pub fn shutdown_telemetry() -> Result<(), Box<dyn std::error::Error>> {
+pub(crate) fn shutdown_telemetry() -> Result<(), Box<dyn std::error::Error>> {
     opentelemetry::global::shutdown_tracer_provider();
     Ok(())
 }
 
 #[cfg(not(feature = "tracing"))]
-pub fn init_telemetry() -> Result<(), Box<dyn std::error::Error>> {
+pub(crate) fn init_telemetry() -> Result<(), Box<dyn std::error::Error>> {
     Ok(())
 }
 
 #[cfg(not(feature = "tracing"))]
-pub fn shutdown_telemetry() -> Result<(), Box<dyn std::error::Error>> {
+pub(crate) fn shutdown_telemetry() -> Result<(), Box<dyn std::error::Error>> {
     Ok(())
 }

src/iter/all_ast_node_iter.rs

@@ -0,0 +1,102 @@
+use std::collections::VecDeque;
+
+use super::ast_node::AstNode;
+use super::ast_node_iter::AstNodeIter;
+
+pub struct AllAstNodeIter<'r, 's> {
+    root: Option<AstNode<'r, 's>>,
+    queue: VecDeque<AstNodeIter<'r, 's>>,
+}
+
+impl<'r, 's> Iterator for AllAstNodeIter<'r, 's> {
+    type Item = AstNode<'r, 's>;
+
+    fn next(&mut self) -> Option<Self::Item> {
+        if let Some(root) = self.root.take() {
+            self.queue.push_back(AstNodeIter::from_ast_node(&root));
+            return Some(root);
+        }
+        while let Some(child) = self.queue.front_mut() {
+            let next_elem_this_iter = match child {
+                AstNodeIter::Document(ref mut i) => i.next(),
+                AstNodeIter::Heading(ref mut i) => i.next(),
+                AstNodeIter::Section(ref mut i) => i.next(),
+                AstNodeIter::Paragraph(ref mut i) => i.next(),
+                AstNodeIter::PlainList(ref mut i) => i.next(),
+                AstNodeIter::PlainListItem(ref mut i) => i.next(),
+                AstNodeIter::GreaterBlock(ref mut i) => i.next(),
+                AstNodeIter::DynamicBlock(ref mut i) => i.next(),
+                AstNodeIter::FootnoteDefinition(ref mut i) => i.next(),
+                AstNodeIter::Comment(ref mut i) => i.next(),
+                AstNodeIter::Drawer(ref mut i) => i.next(),
+                AstNodeIter::PropertyDrawer(ref mut i) => i.next(),
+                AstNodeIter::NodeProperty(ref mut i) => i.next(),
+                AstNodeIter::Table(ref mut i) => i.next(),
+                AstNodeIter::TableRow(ref mut i) => i.next(),
+                AstNodeIter::VerseBlock(ref mut i) => i.next(),
+                AstNodeIter::CommentBlock(ref mut i) => i.next(),
+                AstNodeIter::ExampleBlock(ref mut i) => i.next(),
+                AstNodeIter::ExportBlock(ref mut i) => i.next(),
+                AstNodeIter::SrcBlock(ref mut i) => i.next(),
+                AstNodeIter::Clock(ref mut i) => i.next(),
+                AstNodeIter::DiarySexp(ref mut i) => i.next(),
+                AstNodeIter::Planning(ref mut i) => i.next(),
+                AstNodeIter::FixedWidthArea(ref mut i) => i.next(),
+                AstNodeIter::HorizontalRule(ref mut i) => i.next(),
+                AstNodeIter::Keyword(ref mut i) => i.next(),
+                AstNodeIter::BabelCall(ref mut i) => i.next(),
+                AstNodeIter::LatexEnvironment(ref mut i) => i.next(),
+                AstNodeIter::Bold(ref mut i) => i.next(),
+                AstNodeIter::Italic(ref mut i) => i.next(),
+                AstNodeIter::Underline(ref mut i) => i.next(),
+                AstNodeIter::StrikeThrough(ref mut i) => i.next(),
+                AstNodeIter::Code(ref mut i) => i.next(),
+                AstNodeIter::Verbatim(ref mut i) => i.next(),
+                AstNodeIter::PlainText(ref mut i) => i.next(),
+                AstNodeIter::RegularLink(ref mut i) => i.next(),
+                AstNodeIter::RadioLink(ref mut i) => i.next(),
+                AstNodeIter::RadioTarget(ref mut i) => i.next(),
+                AstNodeIter::PlainLink(ref mut i) => i.next(),
+                AstNodeIter::AngleLink(ref mut i) => i.next(),
+                AstNodeIter::OrgMacro(ref mut i) => i.next(),
+                AstNodeIter::Entity(ref mut i) => i.next(),
+                AstNodeIter::LatexFragment(ref mut i) => i.next(),
+                AstNodeIter::ExportSnippet(ref mut i) => i.next(),
+                AstNodeIter::FootnoteReference(ref mut i) => i.next(),
+                AstNodeIter::Citation(ref mut i) => i.next(),
+                AstNodeIter::CitationReference(ref mut i) => i.next(),
+                AstNodeIter::InlineBabelCall(ref mut i) => i.next(),
+                AstNodeIter::InlineSourceBlock(ref mut i) => i.next(),
+                AstNodeIter::LineBreak(ref mut i) => i.next(),
+                AstNodeIter::Target(ref mut i) => i.next(),
+                AstNodeIter::StatisticsCookie(ref mut i) => i.next(),
+                AstNodeIter::Subscript(ref mut i) => i.next(),
+                AstNodeIter::Superscript(ref mut i) => i.next(),
+                AstNodeIter::TableCell(ref mut i) => i.next(),
+                AstNodeIter::Timestamp(ref mut i) => i.next(),
+            };
+            if let Some(next_elem_this_iter) = next_elem_this_iter {
+                self.queue
+                    .push_back(AstNodeIter::from_ast_node(&next_elem_this_iter));
+                return Some(next_elem_this_iter);
+            } else {
+                self.queue.pop_front();
+            }
+        }
+
+        None
+    }
+}
+
+impl<'r, 's> IntoIterator for AstNode<'r, 's> {
+    type Item = AstNode<'r, 's>;
+
+    type IntoIter = AllAstNodeIter<'r, 's>;
+
+    fn into_iter(self) -> Self::IntoIter {
+        AllAstNodeIter {
+            root: Some(self),
+            queue: VecDeque::new(),
+        }
+    }
+}

src/iter/ast_node.rs

@@ -0,0 +1,251 @@
+use super::macros::to_ast_node;
+use crate::types::AngleLink;
+use crate::types::BabelCall;
+use crate::types::Bold;
+use crate::types::Citation;
+use crate::types::CitationReference;
+use crate::types::Clock;
+use crate::types::Code;
+use crate::types::Comment;
+use crate::types::CommentBlock;
+use crate::types::DiarySexp;
+use crate::types::Document;
+use crate::types::DocumentElement;
+use crate::types::Drawer;
+use crate::types::DynamicBlock;
+use crate::types::Element;
+use crate::types::Entity;
+use crate::types::ExampleBlock;
+use crate::types::ExportBlock;
+use crate::types::ExportSnippet;
+use crate::types::FixedWidthArea;
+use crate::types::FootnoteDefinition;
+use crate::types::FootnoteReference;
+use crate::types::GreaterBlock;
+use crate::types::Heading;
+use crate::types::HorizontalRule;
+use crate::types::InlineBabelCall;
+use crate::types::InlineSourceBlock;
+use crate::types::Italic;
+use crate::types::Keyword;
+use crate::types::LatexEnvironment;
+use crate::types::LatexFragment;
+use crate::types::LineBreak;
+use crate::types::NodeProperty;
+use crate::types::Object;
+use crate::types::OrgMacro;
+use crate::types::Paragraph;
+use crate::types::PlainLink;
+use crate::types::PlainList;
+use crate::types::PlainListItem;
+use crate::types::PlainText;
+use crate::types::Planning;
+use crate::types::PropertyDrawer;
+use crate::types::RadioLink;
+use crate::types::RadioTarget;
+use crate::types::RegularLink;
+use crate::types::Section;
+use crate::types::SrcBlock;
+use crate::types::StatisticsCookie;
+use crate::types::StrikeThrough;
+use crate::types::Subscript;
+use crate::types::Superscript;
+use crate::types::Table;
+use crate::types::TableCell;
+use crate::types::TableRow;
+use crate::types::Target;
+use crate::types::Timestamp;
+use crate::types::Underline;
+use crate::types::Verbatim;
+use crate::types::VerseBlock;
+
+pub enum AstNode<'r, 's> {
+    // Document Nodes
+    Document(&'r Document<'s>),
+    Heading(&'r Heading<'s>),
+    Section(&'r Section<'s>),
+    // Elements
+    Paragraph(&'r Paragraph<'s>),
+    PlainList(&'r PlainList<'s>),
+    PlainListItem(&'r PlainListItem<'s>),
+    GreaterBlock(&'r GreaterBlock<'s>),
+    DynamicBlock(&'r DynamicBlock<'s>),
+    FootnoteDefinition(&'r FootnoteDefinition<'s>),
+    Comment(&'r Comment<'s>),
+    Drawer(&'r Drawer<'s>),
+    PropertyDrawer(&'r PropertyDrawer<'s>),
+    NodeProperty(&'r NodeProperty<'s>),
+    Table(&'r Table<'s>),
+    TableRow(&'r TableRow<'s>),
+    VerseBlock(&'r VerseBlock<'s>),
+    CommentBlock(&'r CommentBlock<'s>),
+    ExampleBlock(&'r ExampleBlock<'s>),
+    ExportBlock(&'r ExportBlock<'s>),
+    SrcBlock(&'r SrcBlock<'s>),
+    Clock(&'r Clock<'s>),
+    DiarySexp(&'r DiarySexp<'s>),
+    Planning(&'r Planning<'s>),
+    FixedWidthArea(&'r FixedWidthArea<'s>),
+    HorizontalRule(&'r HorizontalRule<'s>),
+    Keyword(&'r Keyword<'s>),
+    BabelCall(&'r BabelCall<'s>),
+    LatexEnvironment(&'r LatexEnvironment<'s>),
+    // Objects
+    Bold(&'r Bold<'s>),
+    Italic(&'r Italic<'s>),
+    Underline(&'r Underline<'s>),
+    StrikeThrough(&'r StrikeThrough<'s>),
+    Code(&'r Code<'s>),
+    Verbatim(&'r Verbatim<'s>),
+    PlainText(&'r PlainText<'s>),
+    RegularLink(&'r RegularLink<'s>),
+    RadioLink(&'r RadioLink<'s>),
+    RadioTarget(&'r RadioTarget<'s>),
+    PlainLink(&'r PlainLink<'s>),
+    AngleLink(&'r AngleLink<'s>),
+    OrgMacro(&'r OrgMacro<'s>),
+    Entity(&'r Entity<'s>),
+    LatexFragment(&'r LatexFragment<'s>),
+    ExportSnippet(&'r ExportSnippet<'s>),
+    FootnoteReference(&'r FootnoteReference<'s>),
+    Citation(&'r Citation<'s>),
+    CitationReference(&'r CitationReference<'s>),
+    InlineBabelCall(&'r InlineBabelCall<'s>),
+    InlineSourceBlock(&'r InlineSourceBlock<'s>),
+    LineBreak(&'r LineBreak<'s>),
+    Target(&'r Target<'s>),
+    StatisticsCookie(&'r StatisticsCookie<'s>),
+    Subscript(&'r Subscript<'s>),
+    Superscript(&'r Superscript<'s>),
+    TableCell(&'r TableCell<'s>),
+    Timestamp(&'r Timestamp<'s>),
+}
+
+impl<'r, 's> From<&'r DocumentElement<'s>> for AstNode<'r, 's> {
+    fn from(value: &'r DocumentElement<'s>) -> Self {
+        match value {
+            DocumentElement::Heading(inner) => inner.into(),
+            DocumentElement::Section(inner) => inner.into(),
+        }
+    }
+}
+
+impl<'r, 's> From<&'r Element<'s>> for AstNode<'r, 's> {
+    fn from(value: &'r Element<'s>) -> Self {
+        match value {
+            Element::Paragraph(inner) => inner.into(),
+            Element::PlainList(inner) => inner.into(),
+            Element::GreaterBlock(inner) => inner.into(),
+            Element::DynamicBlock(inner) => inner.into(),
+            Element::FootnoteDefinition(inner) => inner.into(),
+            Element::Comment(inner) => inner.into(),
+            Element::Drawer(inner) => inner.into(),
+            Element::PropertyDrawer(inner) => inner.into(),
+            Element::Table(inner) => inner.into(),
+            Element::VerseBlock(inner) => inner.into(),
+            Element::CommentBlock(inner) => inner.into(),
+            Element::ExampleBlock(inner) => inner.into(),
+            Element::ExportBlock(inner) => inner.into(),
+            Element::SrcBlock(inner) => inner.into(),
+            Element::Clock(inner) => inner.into(),
+            Element::DiarySexp(inner) => inner.into(),
+            Element::Planning(inner) => inner.into(),
+            Element::FixedWidthArea(inner) => inner.into(),
+            Element::HorizontalRule(inner) => inner.into(),
+            Element::Keyword(inner) => inner.into(),
+            Element::BabelCall(inner) => inner.into(),
+            Element::LatexEnvironment(inner) => inner.into(),
+        }
+    }
+}
+
+impl<'r, 's> From<&'r Object<'s>> for AstNode<'r, 's> {
+    fn from(value: &'r Object<'s>) -> Self {
+        match value {
+            Object::Bold(inner) => inner.into(),
+            Object::Italic(inner) => inner.into(),
+            Object::Underline(inner) => inner.into(),
+            Object::StrikeThrough(inner) => inner.into(),
+            Object::Code(inner) => inner.into(),
+            Object::Verbatim(inner) => inner.into(),
+            Object::PlainText(inner) => inner.into(),
+            Object::RegularLink(inner) => inner.into(),
+            Object::RadioLink(inner) => inner.into(),
+            Object::RadioTarget(inner) => inner.into(),
+            Object::PlainLink(inner) => inner.into(),
+            Object::AngleLink(inner) => inner.into(),
+            Object::OrgMacro(inner) => inner.into(),
+            Object::Entity(inner) => inner.into(),
+            Object::LatexFragment(inner) => inner.into(),
+            Object::ExportSnippet(inner) => inner.into(),
+            Object::FootnoteReference(inner) => inner.into(),
+            Object::Citation(inner) => inner.into(),
+            Object::CitationReference(inner) => inner.into(),
+            Object::InlineBabelCall(inner) => inner.into(),
+            Object::InlineSourceBlock(inner) => inner.into(),
+            Object::LineBreak(inner) => inner.into(),
+            Object::Target(inner) => inner.into(),
+            Object::StatisticsCookie(inner) => inner.into(),
+            Object::Subscript(inner) => inner.into(),
+            Object::Superscript(inner) => inner.into(),
+            Object::Timestamp(inner) => inner.into(),
+        }
+    }
+}
+
+to_ast_node!(&'r Document<'s>, AstNode::Document);
+to_ast_node!(&'r Heading<'s>, AstNode::Heading);
+to_ast_node!(&'r Section<'s>, AstNode::Section);
+to_ast_node!(&'r Paragraph<'s>, AstNode::Paragraph);
+to_ast_node!(&'r PlainList<'s>, AstNode::PlainList);
+to_ast_node!(&'r PlainListItem<'s>, AstNode::PlainListItem);
+to_ast_node!(&'r GreaterBlock<'s>, AstNode::GreaterBlock);
+to_ast_node!(&'r DynamicBlock<'s>, AstNode::DynamicBlock);
+to_ast_node!(&'r FootnoteDefinition<'s>, AstNode::FootnoteDefinition);
+to_ast_node!(&'r Comment<'s>, AstNode::Comment);
+to_ast_node!(&'r Drawer<'s>, AstNode::Drawer);
+to_ast_node!(&'r PropertyDrawer<'s>, AstNode::PropertyDrawer);
+to_ast_node!(&'r NodeProperty<'s>, AstNode::NodeProperty);
+to_ast_node!(&'r Table<'s>, AstNode::Table);
+to_ast_node!(&'r TableRow<'s>, AstNode::TableRow);
+to_ast_node!(&'r VerseBlock<'s>, AstNode::VerseBlock);
+to_ast_node!(&'r CommentBlock<'s>, AstNode::CommentBlock);
+to_ast_node!(&'r ExampleBlock<'s>, AstNode::ExampleBlock);
+to_ast_node!(&'r ExportBlock<'s>, AstNode::ExportBlock);
+to_ast_node!(&'r SrcBlock<'s>, AstNode::SrcBlock);
+to_ast_node!(&'r Clock<'s>, AstNode::Clock);
+to_ast_node!(&'r DiarySexp<'s>, AstNode::DiarySexp);
+to_ast_node!(&'r Planning<'s>, AstNode::Planning);
+to_ast_node!(&'r FixedWidthArea<'s>, AstNode::FixedWidthArea);
+to_ast_node!(&'r HorizontalRule<'s>, AstNode::HorizontalRule);
+to_ast_node!(&'r Keyword<'s>, AstNode::Keyword);
+to_ast_node!(&'r BabelCall<'s>, AstNode::BabelCall);
+to_ast_node!(&'r LatexEnvironment<'s>, AstNode::LatexEnvironment);
+to_ast_node!(&'r Bold<'s>, AstNode::Bold);
+to_ast_node!(&'r Italic<'s>, AstNode::Italic);
+to_ast_node!(&'r Underline<'s>, AstNode::Underline);
+to_ast_node!(&'r StrikeThrough<'s>, AstNode::StrikeThrough);
+to_ast_node!(&'r Code<'s>, AstNode::Code);
+to_ast_node!(&'r Verbatim<'s>, AstNode::Verbatim);
+to_ast_node!(&'r PlainText<'s>, AstNode::PlainText);
+to_ast_node!(&'r RegularLink<'s>, AstNode::RegularLink);
+to_ast_node!(&'r RadioLink<'s>, AstNode::RadioLink);
+to_ast_node!(&'r RadioTarget<'s>, AstNode::RadioTarget);
+to_ast_node!(&'r PlainLink<'s>, AstNode::PlainLink);
+to_ast_node!(&'r AngleLink<'s>, AstNode::AngleLink);
+to_ast_node!(&'r OrgMacro<'s>, AstNode::OrgMacro);
+to_ast_node!(&'r Entity<'s>, AstNode::Entity);
+to_ast_node!(&'r LatexFragment<'s>, AstNode::LatexFragment);
+to_ast_node!(&'r ExportSnippet<'s>, AstNode::ExportSnippet);
+to_ast_node!(&'r FootnoteReference<'s>, AstNode::FootnoteReference);
+to_ast_node!(&'r Citation<'s>, AstNode::Citation);
+to_ast_node!(&'r CitationReference<'s>, AstNode::CitationReference);
+to_ast_node!(&'r InlineBabelCall<'s>, AstNode::InlineBabelCall);
+to_ast_node!(&'r InlineSourceBlock<'s>, AstNode::InlineSourceBlock);
+to_ast_node!(&'r LineBreak<'s>, AstNode::LineBreak);
+to_ast_node!(&'r Target<'s>, AstNode::Target);
+to_ast_node!(&'r StatisticsCookie<'s>, AstNode::StatisticsCookie);
+to_ast_node!(&'r Subscript<'s>, AstNode::Subscript);
+to_ast_node!(&'r Superscript<'s>, AstNode::Superscript);
+to_ast_node!(&'r TableCell<'s>, AstNode::TableCell);
+to_ast_node!(&'r Timestamp<'s>, AstNode::Timestamp);

src/iter/ast_node_iter.rs

@@ -0,0 +1,332 @@
+use std::marker::PhantomData;
+
+use super::ast_node::AstNode;
+use super::macros::children_iter;
+use super::macros::empty_iter;
+use super::macros::multi_field_iter;
+use crate::types::AngleLink;
+use crate::types::BabelCall;
+use crate::types::Bold;
+use crate::types::Citation;
+use crate::types::CitationReference;
+use crate::types::Clock;
+use crate::types::Code;
+use crate::types::Comment;
+use crate::types::CommentBlock;
+use crate::types::DiarySexp;
+use crate::types::Document;
+use crate::types::DocumentElement;
+use crate::types::Drawer;
+use crate::types::DynamicBlock;
+use crate::types::Element;
+use crate::types::Entity;
+use crate::types::ExampleBlock;
+use crate::types::ExportBlock;
+use crate::types::ExportSnippet;
+use crate::types::FixedWidthArea;
+use crate::types::FootnoteDefinition;
+use crate::types::FootnoteReference;
+use crate::types::GreaterBlock;
+use crate::types::Heading;
+use crate::types::HorizontalRule;
+use crate::types::InlineBabelCall;
+use crate::types::InlineSourceBlock;
+use crate::types::Italic;
+use crate::types::Keyword;
+use crate::types::LatexEnvironment;
+use crate::types::LatexFragment;
+use crate::types::LineBreak;
+use crate::types::NodeProperty;
+use crate::types::Object;
+use crate::types::OrgMacro;
+use crate::types::Paragraph;
+use crate::types::PlainLink;
+use crate::types::PlainList;
+use crate::types::PlainListItem;
+use crate::types::PlainText;
+use crate::types::Planning;
+use crate::types::PropertyDrawer;
+use crate::types::RadioLink;
+use crate::types::RadioTarget;
+use crate::types::RegularLink;
+use crate::types::Section;
+use crate::types::SrcBlock;
+use crate::types::StatisticsCookie;
+use crate::types::StrikeThrough;
+use crate::types::Subscript;
+use crate::types::Superscript;
+use crate::types::Table;
+use crate::types::TableCell;
+use crate::types::TableRow;
+use crate::types::Target;
+use crate::types::Timestamp;
+use crate::types::Underline;
+use crate::types::Verbatim;
+use crate::types::VerseBlock;
+
+/// Iterator over the AST nodes contained within the starting node.
+///
+/// This only iterates over the children, not the starting node itself. So an AstNodeIter::PlainList would only return PlainListItems, not the PlainList.
+///
+/// This only iterates over AST nodes, so an AstNodeIter::Heading would iterate over both the title and section contents, but it would not iterate over simple strings like the TODO keyword or priority.
+pub(crate) enum AstNodeIter<'r, 's> {
+    // Document Nodes
+    Document(DocumentIter<'r, 's>),
+    Heading(HeadingIter<'r, 's>),
+    Section(SectionIter<'r, 's>),
+    // Elements
+    Paragraph(ParagraphIter<'r, 's>),
+    PlainList(PlainListIter<'r, 's>),
+    PlainListItem(PlainListItemIter<'r, 's>),
+    GreaterBlock(GreaterBlockIter<'r, 's>),
+    DynamicBlock(DynamicBlockIter<'r, 's>),
+    FootnoteDefinition(FootnoteDefinitionIter<'r, 's>),
+    Comment(CommentIter<'r, 's>),
+    Drawer(DrawerIter<'r, 's>),
+    PropertyDrawer(PropertyDrawerIter<'r, 's>),
+    NodeProperty(NodePropertyIter<'r, 's>),
+    Table(TableIter<'r, 's>),
+    TableRow(TableRowIter<'r, 's>),
+    VerseBlock(VerseBlockIter<'r, 's>),
+    CommentBlock(CommentBlockIter<'r, 's>),
+    ExampleBlock(ExampleBlockIter<'r, 's>),
+    ExportBlock(ExportBlockIter<'r, 's>),
+    SrcBlock(SrcBlockIter<'r, 's>),
+    Clock(ClockIter<'r, 's>),
+    DiarySexp(DiarySexpIter<'r, 's>),
+    Planning(PlanningIter<'r, 's>),
+    FixedWidthArea(FixedWidthAreaIter<'r, 's>),
+    HorizontalRule(HorizontalRuleIter<'r, 's>),
+    Keyword(KeywordIter<'r, 's>),
+    BabelCall(BabelCallIter<'r, 's>),
+    LatexEnvironment(LatexEnvironmentIter<'r, 's>),
+    // Objects
+    Bold(BoldIter<'r, 's>),
+    Italic(ItalicIter<'r, 's>),
+    Underline(UnderlineIter<'r, 's>),
+    StrikeThrough(StrikeThroughIter<'r, 's>),
+    Code(CodeIter<'r, 's>),
+    Verbatim(VerbatimIter<'r, 's>),
+    PlainText(PlainTextIter<'r, 's>),
+    RegularLink(RegularLinkIter<'r, 's>),
+    RadioLink(RadioLinkIter<'r, 's>),
+    RadioTarget(RadioTargetIter<'r, 's>),
+    PlainLink(PlainLinkIter<'r, 's>),
+    AngleLink(AngleLinkIter<'r, 's>),
+    OrgMacro(OrgMacroIter<'r, 's>),
+    Entity(EntityIter<'r, 's>),
+    LatexFragment(LatexFragmentIter<'r, 's>),
+    ExportSnippet(ExportSnippetIter<'r, 's>),
+    FootnoteReference(FootnoteReferenceIter<'r, 's>),
+    Citation(CitationIter<'r, 's>),
+    CitationReference(CitationReferenceIter<'r, 's>),
+    InlineBabelCall(InlineBabelCallIter<'r, 's>),
+    InlineSourceBlock(InlineSourceBlockIter<'r, 's>),
+    LineBreak(LineBreakIter<'r, 's>),
+    Target(TargetIter<'r, 's>),
+    StatisticsCookie(StatisticsCookieIter<'r, 's>),
+    Subscript(SubscriptIter<'r, 's>),
+    Superscript(SuperscriptIter<'r, 's>),
+    TableCell(TableCellIter<'r, 's>),
+    Timestamp(TimestampIter<'r, 's>),
+}
+
+impl<'r, 's> AstNodeIter<'r, 's> {
+    pub(crate) fn from_ast_node(node: &AstNode<'r, 's>) -> AstNodeIter<'r, 's> {
+        match node {
+            AstNode::Document(inner) => AstNodeIter::Document(inner.into_iter()),
+            AstNode::Heading(inner) => AstNodeIter::Heading(inner.into_iter()),
+            AstNode::Section(inner) => AstNodeIter::Section(inner.into_iter()),
+            AstNode::Paragraph(inner) => AstNodeIter::Paragraph(inner.into_iter()),
+            AstNode::PlainList(inner) => AstNodeIter::PlainList(inner.into_iter()),
+            AstNode::PlainListItem(inner) => AstNodeIter::PlainListItem(inner.into_iter()),
+            AstNode::GreaterBlock(inner) => AstNodeIter::GreaterBlock(inner.into_iter()),
+            AstNode::DynamicBlock(inner) => AstNodeIter::DynamicBlock(inner.into_iter()),
+            AstNode::FootnoteDefinition(inner) => {
+                AstNodeIter::FootnoteDefinition(inner.into_iter())
+            }
+            AstNode::Comment(inner) => AstNodeIter::Comment(inner.into_iter()),
+            AstNode::Drawer(inner) => AstNodeIter::Drawer(inner.into_iter()),
+            AstNode::PropertyDrawer(inner) => AstNodeIter::PropertyDrawer(inner.into_iter()),
+            AstNode::NodeProperty(inner) => AstNodeIter::NodeProperty(inner.into_iter()),
+            AstNode::Table(inner) => AstNodeIter::Table(inner.into_iter()),
+            AstNode::TableRow(inner) => AstNodeIter::TableRow(inner.into_iter()),
+            AstNode::VerseBlock(inner) => AstNodeIter::VerseBlock(inner.into_iter()),
+            AstNode::CommentBlock(inner) => AstNodeIter::CommentBlock(inner.into_iter()),
+            AstNode::ExampleBlock(inner) => AstNodeIter::ExampleBlock(inner.into_iter()),
+            AstNode::ExportBlock(inner) => AstNodeIter::ExportBlock(inner.into_iter()),
+            AstNode::SrcBlock(inner) => AstNodeIter::SrcBlock(inner.into_iter()),
+            AstNode::Clock(inner) => AstNodeIter::Clock(inner.into_iter()),
+            AstNode::DiarySexp(inner) => AstNodeIter::DiarySexp(inner.into_iter()),
+            AstNode::Planning(inner) => AstNodeIter::Planning(inner.into_iter()),
+            AstNode::FixedWidthArea(inner) => AstNodeIter::FixedWidthArea(inner.into_iter()),
+            AstNode::HorizontalRule(inner) => AstNodeIter::HorizontalRule(inner.into_iter()),
+            AstNode::Keyword(inner) => AstNodeIter::Keyword(inner.into_iter()),
+            AstNode::BabelCall(inner) => AstNodeIter::BabelCall(inner.into_iter()),
+            AstNode::LatexEnvironment(inner) => AstNodeIter::LatexEnvironment(inner.into_iter()),
+            AstNode::Bold(inner) => AstNodeIter::Bold(inner.into_iter()),
+            AstNode::Italic(inner) => AstNodeIter::Italic(inner.into_iter()),
+            AstNode::Underline(inner) => AstNodeIter::Underline(inner.into_iter()),
+            AstNode::StrikeThrough(inner) => AstNodeIter::StrikeThrough(inner.into_iter()),
+            AstNode::Code(inner) => AstNodeIter::Code(inner.into_iter()),
+            AstNode::Verbatim(inner) => AstNodeIter::Verbatim(inner.into_iter()),
+            AstNode::PlainText(inner) => AstNodeIter::PlainText(inner.into_iter()),
+            AstNode::RegularLink(inner) => AstNodeIter::RegularLink(inner.into_iter()),
+            AstNode::RadioLink(inner) => AstNodeIter::RadioLink(inner.into_iter()),
+            AstNode::RadioTarget(inner) => AstNodeIter::RadioTarget(inner.into_iter()),
+            AstNode::PlainLink(inner) => AstNodeIter::PlainLink(inner.into_iter()),
+            AstNode::AngleLink(inner) => AstNodeIter::AngleLink(inner.into_iter()),
+            AstNode::OrgMacro(inner) => AstNodeIter::OrgMacro(inner.into_iter()),
+            AstNode::Entity(inner) => AstNodeIter::Entity(inner.into_iter()),
+            AstNode::LatexFragment(inner) => AstNodeIter::LatexFragment(inner.into_iter()),
+            AstNode::ExportSnippet(inner) => AstNodeIter::ExportSnippet(inner.into_iter()),
+            AstNode::FootnoteReference(inner) => AstNodeIter::FootnoteReference(inner.into_iter()),
+            AstNode::Citation(inner) => AstNodeIter::Citation(inner.into_iter()),
+            AstNode::CitationReference(inner) => AstNodeIter::CitationReference(inner.into_iter()),
+            AstNode::InlineBabelCall(inner) => AstNodeIter::InlineBabelCall(inner.into_iter()),
+            AstNode::InlineSourceBlock(inner) => AstNodeIter::InlineSourceBlock(inner.into_iter()),
+            AstNode::LineBreak(inner) => AstNodeIter::LineBreak(inner.into_iter()),
+            AstNode::Target(inner) => AstNodeIter::Target(inner.into_iter()),
+            AstNode::StatisticsCookie(inner) => AstNodeIter::StatisticsCookie(inner.into_iter()),
+            AstNode::Subscript(inner) => AstNodeIter::Subscript(inner.into_iter()),
+            AstNode::Superscript(inner) => AstNodeIter::Superscript(inner.into_iter()),
+            AstNode::TableCell(inner) => AstNodeIter::TableCell(inner.into_iter()),
+            AstNode::Timestamp(inner) => AstNodeIter::Timestamp(inner.into_iter()),
+        }
+    }
+}
+
+multi_field_iter!(
+    Document<'s>,
+    DocumentIter,
+    zeroth_section,
+    std::option::Iter<'r, Section<'s>>,
+    children,
+    std::slice::Iter<'r, Heading<'s>>
+);
+multi_field_iter!(
+    Heading<'s>,
+    HeadingIter,
+    title,
+    std::slice::Iter<'r, Object<'s>>,
+    children,
+    std::slice::Iter<'r, DocumentElement<'s>>
+);
+children_iter!(Section<'s>, SectionIter, std::slice::Iter<'r, Element<'s>>);
+children_iter!(
+    Paragraph<'s>,
+    ParagraphIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+children_iter!(
+    PlainList<'s>,
+    PlainListIter,
+    std::slice::Iter<'r, PlainListItem<'s>>
+);
+multi_field_iter!(
+    PlainListItem<'s>,
+    PlainListItemIter,
+    tag,
+    std::slice::Iter<'r, Object<'s>>,
+    children,
+    std::slice::Iter<'r, Element<'s>>
+);
+children_iter!(
+    GreaterBlock<'s>,
+    GreaterBlockIter,
+    std::slice::Iter<'r, Element<'s>>
+);
+children_iter!(
+    DynamicBlock<'s>,
+    DynamicBlockIter,
+    std::slice::Iter<'r, Element<'s>>
+);
+children_iter!(
+    FootnoteDefinition<'s>,
+    FootnoteDefinitionIter,
+    std::slice::Iter<'r, Element<'s>>
+);
+empty_iter!(Comment<'s>, CommentIter);
+children_iter!(Drawer<'s>, DrawerIter, std::slice::Iter<'r, Element<'s>>);
+children_iter!(
+    PropertyDrawer<'s>,
+    PropertyDrawerIter,
+    std::slice::Iter<'r, NodeProperty<'s>>
+);
+empty_iter!(NodeProperty<'s>, NodePropertyIter);
+children_iter!(Table<'s>, TableIter, std::slice::Iter<'r, TableRow<'s>>);
+children_iter!(
+    TableRow<'s>,
+    TableRowIter,
+    std::slice::Iter<'r, TableCell<'s>>
+);
+children_iter!(
+    VerseBlock<'s>,
+    VerseBlockIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+empty_iter!(CommentBlock<'s>, CommentBlockIter);
+empty_iter!(ExampleBlock<'s>, ExampleBlockIter);
+empty_iter!(ExportBlock<'s>, ExportBlockIter);
+empty_iter!(SrcBlock<'s>, SrcBlockIter);
+empty_iter!(Clock<'s>, ClockIter);
+empty_iter!(DiarySexp<'s>, DiarySexpIter);
+empty_iter!(Planning<'s>, PlanningIter);
+empty_iter!(FixedWidthArea<'s>, FixedWidthAreaIter);
+empty_iter!(HorizontalRule<'s>, HorizontalRuleIter);
+empty_iter!(Keyword<'s>, KeywordIter);
+empty_iter!(BabelCall<'s>, BabelCallIter);
+empty_iter!(LatexEnvironment<'s>, LatexEnvironmentIter);
+children_iter!(Bold<'s>, BoldIter, std::slice::Iter<'r, Object<'s>>);
+children_iter!(Italic<'s>, ItalicIter, std::slice::Iter<'r, Object<'s>>);
+children_iter!(
+    Underline<'s>,
+    UnderlineIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+children_iter!(
+    StrikeThrough<'s>,
+    StrikeThroughIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+empty_iter!(Code<'s>, CodeIter);
+empty_iter!(Verbatim<'s>, VerbatimIter);
+empty_iter!(PlainText<'s>, PlainTextIter);
+empty_iter!(RegularLink<'s>, RegularLinkIter);
+children_iter!(
+    RadioLink<'s>,
+    RadioLinkIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+children_iter!(
+    RadioTarget<'s>,
+    RadioTargetIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+empty_iter!(PlainLink<'s>, PlainLinkIter);
+empty_iter!(AngleLink<'s>, AngleLinkIter);
+empty_iter!(OrgMacro<'s>, OrgMacroIter);
+empty_iter!(Entity<'s>, EntityIter);
+empty_iter!(LatexFragment<'s>, LatexFragmentIter);
+empty_iter!(ExportSnippet<'s>, ExportSnippetIter);
+multi_field_iter!(
+    FootnoteReference<'s>,
+    FootnoteReferenceIter,
+    definition,
+    std::slice::Iter<'r, Object<'s>>,
+);
+empty_iter!(Citation<'s>, CitationIter);
+empty_iter!(CitationReference<'s>, CitationReferenceIter);
+empty_iter!(InlineBabelCall<'s>, InlineBabelCallIter);
+empty_iter!(InlineSourceBlock<'s>, InlineSourceBlockIter);
+empty_iter!(LineBreak<'s>, LineBreakIter);
+empty_iter!(Target<'s>, TargetIter);
+empty_iter!(StatisticsCookie<'s>, StatisticsCookieIter);
+empty_iter!(Subscript<'s>, SubscriptIter);
+empty_iter!(Superscript<'s>, SuperscriptIter);
+children_iter!(
+    TableCell<'s>,
+    TableCellIter,
+    std::slice::Iter<'r, Object<'s>>
+);
+empty_iter!(Timestamp<'s>, TimestampIter);

src/iter/macros.rs

@@ -0,0 +1,114 @@
+/// Write the implementation of From<> to convert a borrow of the type to an AstNode
+macro_rules! to_ast_node {
+    ($inp:ty, $enum:expr) => {
+        impl<'r, 's> From<$inp> for AstNode<'r, 's> {
+            fn from(value: $inp) -> Self {
+                $enum(value)
+            }
+        }
+    };
+}
+
+pub(crate) use to_ast_node;
+
+/// Create iterators for ast nodes where it only has to iterate over children
+macro_rules! children_iter {
+    ($astnodetype:ty, $itertype:ident, $innertype:ty) => {
+        pub struct $itertype<'r, 's> {
+            next: $innertype,
+        }
+
+        impl<'r, 's> Iterator for $itertype<'r, 's> {
+            type Item = AstNode<'r, 's>;
+
+            fn next(&mut self) -> Option<Self::Item> {
+                self.next.next().map(Into::<AstNode>::into)
+            }
+        }
+
+        impl<'r, 's> IntoIterator for &'r $astnodetype {
+            type Item = AstNode<'r, 's>;
+
+            type IntoIter = $itertype<'r, 's>;
+
+            fn into_iter(self) -> Self::IntoIter {
+                $itertype {
+                    next: self.children.iter(),
+                }
+            }
+        }
+    };
+}
+
+pub(crate) use children_iter;
+
+/// Create iterators for ast nodes that do not contain any ast node children.
+macro_rules! empty_iter {
+    ($astnodetype:ty, $itertype:ident) => {
+        pub struct $itertype<'r, 's> {
+            phantom: PhantomData<&'r $astnodetype>,
+        }
+
+        impl<'r, 's> Iterator for $itertype<'r, 's> {
+            type Item = AstNode<'r, 's>;
+
+            fn next(&mut self) -> Option<Self::Item> {
+                None
+            }
+        }
+
+        impl<'r, 's> IntoIterator for &'r $astnodetype {
+            type Item = AstNode<'r, 's>;
+
+            type IntoIter = $itertype<'r, 's>;
+
+            fn into_iter(self) -> Self::IntoIter {
+                $itertype {
+                    phantom: PhantomData,
+                }
+            }
+        }
+    };
+}
+
+pub(crate) use empty_iter;
+
+/// Create iterators for ast nodes where it has to iterate over multiple child lists.
+macro_rules! multi_field_iter {
+    ($astnodetype:ty, $itertype:ident, $firstfieldname: ident, $firstinnertype:ty, $($fieldname: ident, $innertype:ty),*) => {
+        pub struct $itertype<'r, 's> {
+            $firstfieldname: $firstinnertype,
+            $(
+$fieldname: $innertype,
+),*
+        }
+
+        impl<'r, 's> Iterator for $itertype<'r, 's> {
+            type Item = AstNode<'r, 's>;
+
+            fn next(&mut self) -> Option<Self::Item> {
+                self.$firstfieldname.next().map(Into::<AstNode>::into)
+                $(
+.or_else(|| self.$fieldname.next().map(Into::<AstNode>::into))
+),*
+            }
+        }
+
+        impl<'r, 's> IntoIterator for &'r $astnodetype {
+            type Item = AstNode<'r, 's>;
+
+            type IntoIter = $itertype<'r, 's>;
+
+            fn into_iter(self) -> Self::IntoIter {
+                $itertype {
+            $firstfieldname: self.$firstfieldname.iter(),
+            $(
+$fieldname: self.$fieldname.iter(),
+),*
+                }
+            }
+        }
+    };
+}
+
+pub(crate) use multi_field_iter;

src/iter/mod.rs

@@ -0,0 +1,5 @@
+mod all_ast_node_iter;
+mod ast_node;
+mod ast_node_iter;
+mod macros;
+pub(crate) use ast_node::AstNode;

src/lib.rs

@@ -1,24 +1,17 @@
 #![feature(round_char_boundary)]
 #![feature(exit_status_error)]
 #![feature(trait_alias)]
+// TODO: #![warn(missing_docs)]
 
 #[cfg(feature = "compare")]
-mod compare;
-#[cfg(feature = "compare")]
-pub use compare::compare_document;
-#[cfg(feature = "compare")]
-pub use compare::emacs_parse_anonymous_org_document;
-#[cfg(feature = "compare")]
-pub use compare::emacs_parse_file_org_document;
-#[cfg(feature = "compare")]
-pub use compare::get_emacs_version;
-#[cfg(feature = "compare")]
-pub use compare::get_org_mode_version;
+pub mod compare;
 
 mod context;
 mod error;
+mod iter;
 pub mod parser;
 pub mod types;
 
+pub use context::FileAccessInterface;
 pub use context::GlobalSettings;
 pub use context::LocalFileAccessInterface;

src/main.rs

@@ -4,19 +4,7 @@ use std::io::Read;
 use std::path::Path;
 
 use ::organic::parser::parse;
-#[cfg(feature = "compare")]
-use organic::compare_document;
-#[cfg(feature = "compare")]
-use organic::emacs_parse_anonymous_org_document;
-#[cfg(feature = "compare")]
-use organic::emacs_parse_file_org_document;
-#[cfg(feature = "compare")]
-use organic::get_emacs_version;
-#[cfg(feature = "compare")]
-use organic::get_org_mode_version;
 use organic::parser::parse_with_settings;
-#[cfg(feature = "compare")]
-use organic::parser::sexp::sexp_with_padding;
 use organic::GlobalSettings;
 use organic::LocalFileAccessInterface;
 
@@ -66,85 +54,14 @@ fn read_stdin_to_string() -> Result<String, Box<dyn std::error::Error>> {
     Ok(stdin_contents)
 }
 
-#[cfg(feature = "compare")]
 fn run_anonymous_parse<P: AsRef<str>>(org_contents: P) -> Result<(), Box<dyn std::error::Error>> {
-    let org_contents = org_contents.as_ref();
-    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
-    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
-    let rust_parsed = parse(org_contents)?;
-    let org_sexp = emacs_parse_anonymous_org_document(org_contents)?;
-    let (_remaining, parsed_sexp) =
-        sexp_with_padding(org_sexp.as_str()).map_err(|e| e.to_string())?;
-
-    println!("{}\n\n\n", org_contents);
-    println!("{}", org_sexp);
-    println!("{:#?}", rust_parsed);
-
-    // We do the diffing after printing out both parsed forms in case the diffing panics
-    let diff_result = compare_document(&parsed_sexp, &rust_parsed)?;
-    diff_result.print(org_contents)?;
-
-    if diff_result.is_bad() {
-        Err("Diff results do not match.")?;
-    }
-
-    Ok(())
-}
-
-#[cfg(not(feature = "compare"))]
-fn run_anonymous_parse<P: AsRef<str>>(org_contents: P) -> Result<(), Box<dyn std::error::Error>> {
-    eprintln!(
-        "This program was built with compare disabled. Only parsing with organic, not comparing."
-    );
     let rust_parsed = parse(org_contents.as_ref())?;
     println!("{:#?}", rust_parsed);
     Ok(())
 }
 
-#[cfg(feature = "compare")]
 fn run_parse_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn std::error::Error>> {
     let org_path = org_path.as_ref();
-    eprintln!("Using emacs version: {}", get_emacs_version()?.trim());
-    eprintln!("Using org-mode version: {}", get_org_mode_version()?.trim());
-    let parent_directory = org_path
-        .parent()
-        .ok_or("Should be contained inside a directory.")?;
-    let org_contents = std::fs::read_to_string(org_path)?;
-    let org_contents = org_contents.as_str();
-    let file_access_interface = LocalFileAccessInterface {
-        working_directory: Some(parent_directory.to_path_buf()),
-    };
-    let global_settings = {
-        let mut global_settings = GlobalSettings::default();
-        global_settings.file_access = &file_access_interface;
-        global_settings
-    };
-    let rust_parsed = parse_with_settings(org_contents, &global_settings)?;
-    let org_sexp = emacs_parse_file_org_document(org_path)?;
-    let (_remaining, parsed_sexp) =
-        sexp_with_padding(org_sexp.as_str()).map_err(|e| e.to_string())?;
-
-    println!("{}\n\n\n", org_contents);
-    println!("{}", org_sexp);
-    println!("{:#?}", rust_parsed);
-
-    // We do the diffing after printing out both parsed forms in case the diffing panics
-    let diff_result = compare_document(&parsed_sexp, &rust_parsed)?;
-    diff_result.print(org_contents)?;
-
-    if diff_result.is_bad() {
-        Err("Diff results do not match.")?;
-    }
-
-    Ok(())
-}
-
-#[cfg(not(feature = "compare"))]
-fn run_parse_on_file<P: AsRef<Path>>(org_path: P) -> Result<(), Box<dyn std::error::Error>> {
-    let org_path = org_path.as_ref();
-    eprintln!(
-        "This program was built with compare disabled. Only parsing with organic, not comparing."
-    );
     let parent_directory = org_path
         .parent()
         .ok_or("Should be contained inside a directory.")?;

src/parser/angle_link.rs

@@ -18,7 +18,7 @@ use crate::parser::util::get_consumed;
 use crate::types::AngleLink;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn angle_link<'b, 'g, 'r, 's>(
+pub(crate) fn angle_link<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, AngleLink<'s>> {

src/parser/citation.rs

@@ -31,7 +31,7 @@ use crate::types::Citation;
 use crate::types::Object;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn citation<'b, 'g, 'r, 's>(
+pub(crate) fn citation<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Citation<'s>> {
@@ -187,7 +187,7 @@ mod tests {
     use crate::context::List;
     use crate::parser::element_parser::element;
     use crate::types::Element;
-    use crate::types::Source;
+    use crate::types::GetStandardProperties;
 
     #[test]
     fn citation_simple() {
@@ -202,7 +202,10 @@ mod tests {
             _ => panic!("Should be a paragraph!"),
         };
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(first_paragraph.get_source(), "[cite:@foo]");
+        assert_eq!(
+            first_paragraph.get_standard_properties().get_source(),
+            "[cite:@foo]"
+        );
         assert_eq!(first_paragraph.children.len(), 1);
         assert_eq!(
             first_paragraph

src/parser/citation_reference.rs

@@ -29,7 +29,7 @@ use crate::types::CitationReference;
 use crate::types::Object;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn citation_reference<'b, 'g, 'r, 's>(
+pub(crate) fn citation_reference<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, CitationReference<'s>> {
@@ -49,7 +49,7 @@ pub fn citation_reference<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn citation_reference_key<'b, 'g, 'r, 's>(
+pub(crate) fn citation_reference_key<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
@@ -161,7 +161,7 @@ fn _key_suffix_end<'b, 'g, 'r, 's>(
     tag(";")(input)
 }
 
-pub fn must_balance_bracket<'s, F, O>(
+pub(crate) fn must_balance_bracket<'s, F, O>(
     mut inner: F,
 ) -> impl FnMut(OrgSource<'s>) -> Res<OrgSource<'s>, O>
 where

src/parser/clock.rs

@@ -20,7 +20,7 @@ use crate::parser::util::start_of_line;
 use crate::types::Clock;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn clock<'b, 'g, 'r, 's>(
+pub(crate) fn clock<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Clock<'s>> {

src/parser/comment.rs

@@ -25,7 +25,7 @@ use crate::parser::util::start_of_line;
 use crate::types::Comment;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn comment<'b, 'g, 'r, 's>(
+pub(crate) fn comment<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Comment<'s>> {
@@ -67,6 +67,17 @@ fn comment_line<'b, 'g, 'r, 's>(
     Ok((remaining, source))
 }
 
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+pub(crate) fn detect_comment<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+    tuple((
+        start_of_line,
+        space0,
+        tag("#"),
+        alt((tag(" "), line_ending, eof)),
+    ))(input)?;
+    Ok((input, ()))
+}
+
 #[cfg(test)]
 mod tests {
     use super::*;

src/parser/diary_sexp.rs

@@ -1,16 +1,11 @@
 use nom::branch::alt;
+use nom::bytes::complete::is_not;
 use nom::bytes::complete::tag;
-use nom::character::complete::anychar;
 use nom::character::complete::line_ending;
-use nom::character::complete::space0;
 use nom::combinator::eof;
-use nom::combinator::opt;
-use nom::combinator::recognize;
-use nom::multi::many_till;
 use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
-use super::sexp::sexp;
 use crate::context::RefContext;
 use crate::error::Res;
 use crate::parser::util::get_consumed;
@@ -18,22 +13,14 @@ use crate::parser::util::start_of_line;
 use crate::types::DiarySexp;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn diary_sexp<'b, 'g, 'r, 's>(
+pub(crate) fn diary_sexp<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, DiarySexp<'s>> {
     start_of_line(input)?;
-    let (remaining, _leading_whitespace) = space0(input)?;
-    let (remaining, _clock) = tag("%%")(remaining)?;
-    let (remaining, _gap_whitespace) = space0(remaining)?;
-    let (remaining, _sexp) = recognize(sexp)(remaining)?;
-    let (remaining, _trailing_comment) = opt(tuple((
-        space0,
-        tag(";"),
-        many_till(anychar, alt((line_ending, eof))),
-    )))(remaining)?;
-    let (remaining, _trailing_whitespace) =
-        recognize(tuple((space0, alt((line_ending, eof)))))(remaining)?;
+    let (remaining, _clock) = tag("%%(")(input)?;
+    let (remaining, _contents) = is_not("\r\n")(remaining)?;
+    let (remaining, _eol) = alt((line_ending, eof))(remaining)?;
 
     let source = get_consumed(input, remaining);
     Ok((
@@ -43,3 +30,9 @@ pub fn diary_sexp<'b, 'g, 'r, 's>(
         },
     ))
 }
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+pub(crate) fn detect_diary_sexp<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+    tuple((start_of_line, tag("%%(")))(input)?;
+    Ok((input, ()))
+}

src/parser/document.rs

@@ -7,8 +7,6 @@ use super::in_buffer_settings::apply_in_buffer_settings;
 use super::in_buffer_settings::scan_for_in_buffer_settings;
 use super::org_source::OrgSource;
 use super::section::zeroth_section;
-use super::token::AllTokensIterator;
-use super::token::Token;
 use super::util::get_consumed;
 use crate::context::parser_with_context;
 use crate::context::Context;
@@ -19,6 +17,7 @@ use crate::context::RefContext;
 use crate::error::CustomError;
 use crate::error::MyError;
 use crate::error::Res;
+use crate::iter::AstNode;
 use crate::parser::org_source::convert_error;
 use crate::parser::util::blank_line;
 use crate::types::Document;
@@ -28,7 +27,7 @@ use crate::types::Object;
 ///
 /// This is the main entry point for Organic. It will parse the full contents of the input string as an org-mode document.
 #[allow(dead_code)]
-pub fn parse<'s>(input: &'s str) -> Result<Document<'s>, String> {
+pub fn parse<'s>(input: &'s str) -> Result<Document<'s>, Box<dyn std::error::Error>> {
     parse_with_settings(input, &GlobalSettings::default())
 }
 
@@ -41,7 +40,7 @@ pub fn parse<'s>(input: &'s str) -> Result<Document<'s>, String> {
 pub fn parse_with_settings<'g, 's>(
     input: &'s str,
     global_settings: &'g GlobalSettings<'g, 's>,
-) -> Result<Document<'s>, String> {
+) -> Result<Document<'s>, Box<dyn std::error::Error>> {
     let initial_context = ContextElement::document_context();
     let initial_context = Context::new(global_settings, List::new(&initial_context));
     let wrapped_input = OrgSource::new(input);
@@ -49,7 +48,7 @@ pub fn parse_with_settings<'g, 's>(
         all_consuming(parser_with_context!(document_org_source)(&initial_context))(wrapped_input)
             .map_err(|err| err.to_string())
             .map(|(_remaining, parsed_document)| parsed_document);
-    ret
+    Ok(ret?)
 }
 
 /// Parse a full org-mode document.
@@ -58,7 +57,7 @@ pub fn parse_with_settings<'g, 's>(
 ///
 /// This will not prevent additional settings from being learned during parsing, for example when encountering a "#+TODO".
 #[allow(dead_code)]
-pub fn document<'b, 'g, 'r, 's>(
+fn document<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: &'s str,
 ) -> Res<&'s str, Document<'s>> {
@@ -111,15 +110,14 @@ fn document_org_source<'b, 'g, 'r, 's>(
         _document(context, input).map(|(rem, out)| (Into::<&str>::into(rem), out))?;
     {
         // If there are radio targets in this document then we need to parse the entire document again with the knowledge of the radio targets.
-        let all_radio_targets: Vec<&Vec<Object<'_>>> = document
-            .iter_tokens()
-            .filter_map(|tkn| match tkn {
-                Token::Object(obj) => Some(obj),
-                _ => None,
-            })
-            .filter_map(|obj| match obj {
-                Object::RadioTarget(rt) => Some(rt),
-                _ => None,
+        let all_radio_targets: Vec<&Vec<Object<'_>>> = Into::<AstNode>::into(&document)
+            .into_iter()
+            .filter_map(|ast_node| {
+                if let AstNode::RadioTarget(ast_node) = ast_node {
+                    Some(ast_node)
+                } else {
+                    None
+                }
             })
             .map(|rt| &rt.children)
             .collect();
@@ -155,9 +153,3 @@ fn _document<'b, 'g, 'r, 's>(
         },
     ))
 }
-
-impl<'s> Document<'s> {
-    pub fn iter_tokens<'r>(&'r self) -> impl Iterator<Item = Token<'r, 's>> {
-        AllTokensIterator::new(Token::Document(self))
-    }
-}

src/parser/drawer.rs

@@ -32,7 +32,7 @@ use crate::types::Paragraph;
 use crate::types::SetSource;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn drawer<'b, 'g, 'r, 's>(
+pub(crate) fn drawer<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Drawer<'s>> {

src/parser/dynamic_block.rs

@@ -1,14 +1,18 @@
 use nom::branch::alt;
 use nom::bytes::complete::is_not;
+use nom::bytes::complete::tag;
 use nom::bytes::complete::tag_no_case;
 use nom::character::complete::line_ending;
 use nom::character::complete::space0;
 use nom::character::complete::space1;
+use nom::combinator::consumed;
 use nom::combinator::eof;
 use nom::combinator::not;
 use nom::combinator::opt;
 use nom::combinator::recognize;
+use nom::multi::many0;
 use nom::multi::many_till;
+use nom::sequence::preceded;
 use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
@@ -32,11 +36,10 @@ use crate::types::Paragraph;
 use crate::types::SetSource;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn dynamic_block<'b, 'g, 'r, 's>(
+pub(crate) fn dynamic_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, DynamicBlock<'s>> {
-    // TODO: Do I need to differentiate between different dynamic block types.
     if immediate_in_section(context, "dynamic block") {
         return Err(nom::Err::Error(CustomError::MyError(MyError(
             "Cannot nest objects of the same element".into(),
@@ -67,24 +70,23 @@ pub fn dynamic_block<'b, 'g, 'r, 's>(
     };
     let element_matcher = parser_with_context!(element(true))(&parser_context);
     let exit_matcher = parser_with_context!(exit_matcher_parser)(&parser_context);
-    let (remaining, children) = match tuple((
-        not(exit_matcher),
+    not(exit_matcher)(remaining)?;
+    let (remaining, leading_blank_lines) = opt(consumed(tuple((
         blank_line,
-        many_till(blank_line, exit_matcher),
-    ))(remaining)
-    {
-        Ok((remain, (_not_immediate_exit, first_line, (_trailing_whitespace, _exit_contents)))) => {
+        many0(preceded(not(exit_matcher), blank_line)),
+    ))))(remaining)?;
+    let leading_blank_lines =
+        leading_blank_lines.map(|(source, (first_line, _remaining_lines))| {
             let mut element = Element::Paragraph(Paragraph::of_text(first_line.into()));
-            let source = get_consumed(remaining, remain);
             element.set_source(source.into());
-            (remain, vec![element])
-        }
-        Err(_) => {
-            let (remaining, (children, _exit_contents)) =
-                many_till(element_matcher, exit_matcher)(remaining)?;
-            (remaining, children)
-        }
-    };
+            element
+        });
+    let (remaining, (mut children, _exit_contents)) =
+        many_till(element_matcher, exit_matcher)(remaining)?;
+    if let Some(lines) = leading_blank_lines {
+        children.insert(0, lines);
+    }
+
     let (remaining, _end) = dynamic_block_end(&parser_context, remaining)?;
 
     let source = get_consumed(input, remaining);
@@ -117,7 +119,8 @@ fn dynamic_block_end<'b, 'g, 'r, 's>(
     start_of_line(input)?;
     let (remaining, source) = recognize(tuple((
         space0,
-        tag_no_case("#+end:"),
+        tag_no_case("#+end"),
+        opt(tag(":")),
         alt((eof, line_ending)),
     )))(input)?;
     Ok((remaining, source))

src/parser/element_parser.rs

@@ -4,10 +4,14 @@ use nom::multi::many0;
 
 use super::clock::clock;
 use super::comment::comment;
+use super::comment::detect_comment;
+use super::diary_sexp::detect_diary_sexp;
 use super::diary_sexp::diary_sexp;
 use super::drawer::drawer;
 use super::dynamic_block::dynamic_block;
+use super::fixed_width_area::detect_fixed_width_area;
 use super::fixed_width_area::fixed_width_area;
+use super::footnote_definition::detect_footnote_definition;
 use super::footnote_definition::footnote_definition;
 use super::greater_block::greater_block;
 use super::horizontal_rule::horizontal_rule;
@@ -24,6 +28,7 @@ use super::org_source::OrgSource;
 use super::paragraph::paragraph;
 use super::plain_list::detect_plain_list;
 use super::plain_list::plain_list;
+use super::table::detect_table;
 use super::util::get_consumed;
 use super::util::maybe_consume_trailing_whitespace_if_not_exiting;
 use crate::context::parser_with_context;
@@ -35,7 +40,7 @@ use crate::parser::table::org_mode_table;
 use crate::types::Element;
 use crate::types::SetSource;
 
-pub const fn element(
+pub(crate) const fn element(
     can_be_paragraph: bool,
 ) -> impl for<'b, 'g, 'r, 's> Fn(
     RefContext<'b, 'g, 'r, 's>,
@@ -102,6 +107,7 @@ fn _element<'b, 'g, 'r, 's>(
                 match map(paragraph_matcher, Element::Paragraph)(remaining) {
                     the_ok @ Ok(_) => the_ok,
                     Err(_) => {
+                        // TODO: Because this function expects a single element, if there are multiple affiliated keywords before an element that cannot have affiliated keywords, we end up re-parsing the affiliated keywords many times.
                         affiliated_keywords.clear();
                         map(affiliated_keyword_matcher, Element::Keyword)(input)
                     }
@@ -122,7 +128,7 @@ fn _element<'b, 'g, 'r, 's>(
     Ok((remaining, element))
 }
 
-pub const fn detect_element(
+pub(crate) const fn detect_element(
     can_be_paragraph: bool,
 ) -> impl for<'b, 'g, 'r, 's> Fn(RefContext<'b, 'g, 'r, 's>, OrgSource<'s>) -> Res<OrgSource<'s>, ()>
 {
@@ -135,7 +141,16 @@ fn _detect_element<'b, 'g, 'r, 's>(
     input: OrgSource<'s>,
     can_be_paragraph: bool,
 ) -> Res<OrgSource<'s>, ()> {
-    if detect_plain_list(input).is_ok() {
+    if alt((
+        parser_with_context!(detect_plain_list)(context),
+        detect_footnote_definition,
+        detect_diary_sexp,
+        detect_comment,
+        detect_fixed_width_area,
+        detect_table,
+    ))(input)
+    .is_ok()
+    {
         return Ok((input, ()));
     }
     if _element(context, input, can_be_paragraph).is_ok() {

src/parser/entity.rs

@@ -1,10 +1,10 @@
 use nom::branch::alt;
 use nom::bytes::complete::tag;
-use nom::bytes::complete::tag_no_case;
 use nom::character::complete::satisfy;
 use nom::combinator::eof;
 use nom::combinator::peek;
 use nom::combinator::recognize;
+use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
 use super::util::maybe_consume_object_trailing_whitespace_if_not_exiting;
@@ -433,13 +433,13 @@ const ORG_ENTITIES: [&'static str; 413] = [
 ];
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn entity<'b, 'g, 'r, 's>(
+pub(crate) fn entity<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Entity<'s>> {
     let (remaining, _) = tag("\\")(input)?;
     let (remaining, entity_name) = name(context, remaining)?;
-    let (remaining, _) = alt((tag("{}"), peek(recognize(entity_end))))(remaining)?;
+
     let (remaining, _trailing_whitespace) =
         maybe_consume_object_trailing_whitespace_if_not_exiting(context, remaining)?;
 
@@ -460,9 +460,12 @@ fn name<'b, 'g, 'r, 's>(
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
     // TODO: This should be defined by org-entities and optionally org-entities-user
     for entity in ORG_ENTITIES {
-        let result = tag_no_case::<_, _, CustomError<_>>(entity)(input);
+        let result = tuple((
+            tag::<_, _, CustomError<_>>(entity),
+            alt((tag("{}"), peek(recognize(entity_end)))),
+        ))(input);
         match result {
-            Ok((remaining, ent)) => {
+            Ok((remaining, (ent, _))) => {
                 return Ok((remaining, ent));
             }
             Err(_) => {}

src/parser/export_snippet.rs

@@ -20,7 +20,7 @@ use crate::parser::util::get_consumed;
 use crate::types::ExportSnippet;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn export_snippet<'b, 'g, 'r, 's>(
+pub(crate) fn export_snippet<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ExportSnippet<'s>> {

src/parser/fixed_width_area.rs

@@ -3,15 +3,16 @@ use nom::bytes::complete::is_not;
 use nom::bytes::complete::tag;
 use nom::character::complete::line_ending;
 use nom::character::complete::space0;
-use nom::character::complete::space1;
 use nom::combinator::eof;
 use nom::combinator::not;
-use nom::combinator::opt;
+use nom::combinator::recognize;
 use nom::multi::many0;
 use nom::sequence::preceded;
 use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
+use super::util::only_space1;
+use super::util::org_line_ending;
 use crate::context::parser_with_context;
 use crate::context::RefContext;
 use crate::error::Res;
@@ -21,7 +22,7 @@ use crate::parser::util::start_of_line;
 use crate::types::FixedWidthArea;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn fixed_width_area<'b, 'g, 'r, 's>(
+pub(crate) fn fixed_width_area<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, FixedWidthArea<'s>> {
@@ -47,11 +48,22 @@ fn fixed_width_area_line<'b, 'g, 'r, 's>(
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
     start_of_line(input)?;
     let (remaining, _indent) = space0(input)?;
-    let (remaining, (_hash, _leading_whitespace_and_content, _line_ending)) = tuple((
+    let (remaining, _) = tuple((
         tag(":"),
-        opt(tuple((space1, is_not("\r\n")))),
-        alt((line_ending, eof)),
+        alt((recognize(tuple((only_space1, is_not("\r\n")))), space0)),
+        org_line_ending,
     ))(remaining)?;
     let source = get_consumed(input, remaining);
     Ok((remaining, source))
 }
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+pub(crate) fn detect_fixed_width_area<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+    tuple((
+        start_of_line,
+        space0,
+        tag(":"),
+        alt((tag(" "), line_ending, eof)),
+    ))(input)?;
+    Ok((input, ()))
+}

src/parser/footnote_definition.rs

@@ -2,7 +2,6 @@ use nom::branch::alt;
 use nom::bytes::complete::tag;
 use nom::bytes::complete::tag_no_case;
 use nom::bytes::complete::take_while;
-use nom::character::complete::digit1;
 use nom::character::complete::space0;
 use nom::combinator::opt;
 use nom::combinator::recognize;
@@ -33,7 +32,7 @@ use crate::parser::util::start_of_line;
 use crate::types::FootnoteDefinition;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn footnote_definition<'b, 'g, 'r, 's>(
+pub(crate) fn footnote_definition<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, FootnoteDefinition<'s>> {
@@ -93,11 +92,8 @@ pub fn footnote_definition<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn label<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    alt((
-        digit1,
-        take_while(|c| WORD_CONSTITUENT_CHARACTERS.contains(c) || "-_".contains(c)),
-    ))(input)
+pub(crate) fn label<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    take_while(|c| WORD_CONSTITUENT_CHARACTERS.contains(c) || "-_".contains(c))(input)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -122,7 +118,7 @@ fn footnote_definition_end<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn detect_footnote_definition<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+pub(crate) fn detect_footnote_definition<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
     tuple((start_of_line, tag_no_case("[fn:"), label, tag("]")))(input)?;
     Ok((input, ()))
 }
@@ -133,7 +129,7 @@ mod tests {
     use crate::context::Context;
     use crate::context::GlobalSettings;
     use crate::context::List;
-    use crate::types::Source;
+    use crate::types::GetStandardProperties;
 
     #[test]
     fn two_paragraphs() {
@@ -154,13 +150,17 @@ line footnote.",
             footnote_definition_matcher(remaining).expect("Parse second footnote_definition.");
         assert_eq!(Into::<&str>::into(remaining), "");
         assert_eq!(
-            first_footnote_definition.get_source(),
+            first_footnote_definition
+                .get_standard_properties()
+                .get_source(),
             "[fn:1] A footnote.
 
 "
         );
         assert_eq!(
-            second_footnote_definition.get_source(),
+            second_footnote_definition
+                .get_standard_properties()
+                .get_source(),
             "[fn:2] A multi-
 
 line footnote."
@@ -185,7 +185,9 @@ not in the footnote.",
             footnote_definition_matcher(input).expect("Parse first footnote_definition");
         assert_eq!(Into::<&str>::into(remaining), "not in the footnote.");
         assert_eq!(
-            first_footnote_definition.get_source(),
+            first_footnote_definition
+                .get_standard_properties()
+                .get_source(),
             "[fn:2] A multi-
 
 line footnote.

src/parser/footnote_reference.rs

@@ -23,7 +23,7 @@ use crate::parser::util::get_consumed;
 use crate::types::FootnoteReference;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn footnote_reference<'b, 'g, 'r, 's>(
+pub(crate) fn footnote_reference<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, FootnoteReference<'s>> {

src/parser/greater_block.rs

@@ -4,11 +4,14 @@ use nom::bytes::complete::tag_no_case;
 use nom::character::complete::line_ending;
 use nom::character::complete::space0;
 use nom::character::complete::space1;
+use nom::combinator::consumed;
 use nom::combinator::eof;
 use nom::combinator::not;
 use nom::combinator::opt;
 use nom::combinator::verify;
+use nom::multi::many0;
 use nom::multi::many_till;
+use nom::sequence::preceded;
 use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
@@ -33,7 +36,7 @@ use crate::types::Paragraph;
 use crate::types::SetSource;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn greater_block<'b, 'g, 'r, 's>(
+pub(crate) fn greater_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, GreaterBlock<'s>> {
@@ -80,25 +83,23 @@ pub fn greater_block<'b, 'g, 'r, 's>(
 
     let element_matcher = parser_with_context!(element(true))(&parser_context);
     let exit_matcher = parser_with_context!(exit_matcher_parser)(&parser_context);
-    // Check for a completely empty block
-    let (remaining, children) = match tuple((
-        not(exit_matcher),
+    not(exit_matcher)(remaining)?;
+    let (remaining, leading_blank_lines) = opt(consumed(tuple((
         blank_line,
-        many_till(blank_line, exit_matcher),
-    ))(remaining)
-    {
-        Ok((remain, (_not_immediate_exit, first_line, (_trailing_whitespace, _exit_contents)))) => {
+        many0(preceded(not(exit_matcher), blank_line)),
+    ))))(remaining)?;
+    let leading_blank_lines =
+        leading_blank_lines.map(|(source, (first_line, _remaining_lines))| {
             let mut element = Element::Paragraph(Paragraph::of_text(first_line.into()));
-            let source = get_consumed(remaining, remain);
             element.set_source(source.into());
-            (remain, vec![element])
-        }
-        Err(_) => {
-            let (remaining, (children, _exit_contents)) =
-                many_till(element_matcher, exit_matcher)(remaining)?;
-            (remaining, children)
-        }
-    };
+            element
+        });
+    let (remaining, (mut children, _exit_contents)) =
+        many_till(element_matcher, exit_matcher)(remaining)?;
+    if let Some(lines) = leading_blank_lines {
+        children.insert(0, lines);
+    }
+
     let (remaining, _end) = exit_with_name(&parser_context, remaining)?;
 
     // Not checking if parent exit matcher is causing exit because the greater_block_end matcher asserts we matched a full greater block
@@ -126,7 +127,6 @@ fn parameters<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
 }
 
 fn greater_block_end<'c>(name: &'c str) -> impl ContextMatcher + 'c {
-    // TODO: Can this be done without making an owned copy?
     move |context, input: OrgSource<'_>| _greater_block_end(context, input, name)
 }
 

src/parser/headline.rs

@@ -1,24 +1,26 @@
 use nom::branch::alt;
+use nom::bytes::complete::is_a;
 use nom::bytes::complete::tag;
 use nom::character::complete::anychar;
-use nom::character::complete::line_ending;
 use nom::character::complete::space0;
 use nom::character::complete::space1;
-use nom::combinator::eof;
 use nom::combinator::map;
 use nom::combinator::not;
 use nom::combinator::opt;
+use nom::combinator::peek;
 use nom::combinator::recognize;
 use nom::combinator::verify;
 use nom::multi::many0;
 use nom::multi::many1;
-use nom::multi::many1_count;
 use nom::multi::separated_list1;
 use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
 use super::section::section;
 use super::util::get_consumed;
+use super::util::org_line_ending;
+use super::util::org_space;
+use super::util::org_space_or_line_ending;
 use super::util::start_of_line;
 use crate::context::parser_with_context;
 use crate::context::ContextElement;
@@ -32,30 +34,39 @@ use crate::parser::object_parser::standard_set_object;
 use crate::parser::util::blank_line;
 use crate::types::DocumentElement;
 use crate::types::Heading;
+use crate::types::HeadlineLevel;
 use crate::types::Object;
 use crate::types::PriorityCookie;
 use crate::types::TodoKeywordType;
 
-pub const fn heading(
-    parent_stars: usize,
+pub(crate) const fn heading(
+    parent_level: HeadlineLevel,
 ) -> impl for<'b, 'g, 'r, 's> Fn(
     RefContext<'b, 'g, 'r, 's>,
     OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Heading<'s>> {
-    move |context, input: OrgSource<'_>| _heading(context, input, parent_stars)
+    move |context, input: OrgSource<'_>| _heading(context, input, parent_level)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
 fn _heading<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
-    parent_stars: usize,
+    parent_star_count: HeadlineLevel,
 ) -> Res<OrgSource<'s>, Heading<'s>> {
     not(|i| context.check_exit_matcher(i))(input)?;
     let (
         remaining,
-        (star_count, maybe_todo_keyword, maybe_priority, maybe_comment, title, heading_tags),
-    ) = headline(context, input, parent_stars)?;
+        (
+            headline_level,
+            star_count,
+            maybe_todo_keyword,
+            maybe_priority,
+            maybe_comment,
+            title,
+            heading_tags,
+        ),
+    ) = headline(context, input, parent_star_count)?;
     let section_matcher = parser_with_context!(section)(context);
     let heading_matcher = parser_with_context!(heading(star_count))(context);
     let (remaining, maybe_section) =
@@ -80,11 +91,11 @@ fn _heading<'b, 'g, 'r, 's>(
         remaining,
         Heading {
             source: source.into(),
-            stars: star_count,
-            todo_keyword: maybe_todo_keyword.map(|((todo_keyword_type, todo_keyword), _ws)| {
+            level: headline_level,
+            todo_keyword: maybe_todo_keyword.map(|(todo_keyword_type, todo_keyword)| {
                 (todo_keyword_type, Into::<&str>::into(todo_keyword))
             }),
-            priority_cookie: maybe_priority.map(|(priority, _)| priority),
+            priority_cookie: maybe_priority.map(|(_, priority)| priority),
             title,
             tags: heading_tags,
             children,
@@ -95,7 +106,7 @@ fn _heading<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn detect_headline<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+pub(crate) fn detect_headline<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
     tuple((start_of_line, many1(tag("*")), space1))(input)?;
     Ok((input, ()))
 }
@@ -104,14 +115,15 @@ pub fn detect_headline<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
 fn headline<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
-    parent_stars: usize,
+    parent_star_count: HeadlineLevel,
 ) -> Res<
     OrgSource<'s>,
     (
-        usize,
-        Option<((TodoKeywordType, OrgSource<'s>), OrgSource<'s>)>,
-        Option<(PriorityCookie, OrgSource<'s>)>,
-        Option<(OrgSource<'s>, OrgSource<'s>)>,
+        HeadlineLevel,
+        HeadlineLevel,
+        Option<(TodoKeywordType, OrgSource<'s>)>,
+        Option<(OrgSource<'s>, PriorityCookie)>,
+        Option<OrgSource<'s>>,
         Vec<Object<'s>>,
         Vec<&'s str>,
     ),
@@ -122,45 +134,47 @@ fn headline<'b, 'g, 'r, 's>(
     });
     let parser_context = context.with_additional_node(&parser_context);
 
-    let (
-        remaining,
-        (
-            _,
-            star_count,
-            _,
-            maybe_todo_keyword,
-            maybe_priority,
-            maybe_comment,
-            title,
-            maybe_tags,
-            _,
-            _,
-        ),
-    ) = tuple((
+    let (remaining, (_, (headline_level, star_count, _), _)) = tuple((
         start_of_line,
-        verify(many1_count(tag("*")), |star_count| {
-            *star_count > parent_stars
-        }),
-        space1,
-        opt(tuple((
-            parser_with_context!(heading_keyword)(&parser_context),
-            space1,
-        ))),
-        opt(tuple((priority_cookie, space1))),
-        opt(tuple((tag("COMMENT"), space1))),
-        many1(parser_with_context!(standard_set_object)(&parser_context)),
-        opt(tuple((space0, tags))),
-        space0,
-        alt((line_ending, eof)),
+        verify(
+            parser_with_context!(headline_level)(&parser_context),
+            |(_, count, _)| *count > parent_star_count,
+        ),
+        peek(org_space),
     ))(input)?;
+
+    let (remaining, maybe_todo_keyword) = opt(tuple((
+        space1,
+        parser_with_context!(heading_keyword)(&parser_context),
+        peek(org_space_or_line_ending),
+    )))(remaining)?;
+
+    let (remaining, maybe_priority) = opt(tuple((space1, priority_cookie)))(remaining)?;
+
+    let (remaining, maybe_comment) = opt(tuple((
+        space1,
+        tag("COMMENT"),
+        peek(org_space_or_line_ending),
+    )))(remaining)?;
+
+    let (remaining, maybe_title) = opt(tuple((
+        space1,
+        many1(parser_with_context!(standard_set_object)(&parser_context)),
+    )))(remaining)?;
+
+    let (remaining, maybe_tags) = opt(tuple((space0, tags)))(remaining)?;
+
+    let (remaining, _) = tuple((space0, org_line_ending))(remaining)?;
+
     Ok((
         remaining,
         (
+            headline_level,
             star_count,
-            maybe_todo_keyword,
+            maybe_todo_keyword.map(|(_, todo, _)| todo),
             maybe_priority,
-            maybe_comment,
-            title,
+            maybe_comment.map(|(_, comment, _)| comment),
+            maybe_title.map(|(_, title)| title).unwrap_or(Vec::new()),
             maybe_tags
                 .map(|(_ws, tags)| {
                     tags.into_iter()
@@ -177,10 +191,7 @@ fn headline_title_end<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    recognize(tuple((
-        opt(tuple((space0, tags, space0))),
-        alt((line_ending, eof)),
-    )))(input)
+    recognize(tuple((space0, opt(tuple((tags, space0))), org_line_ending)))(input)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -256,3 +267,23 @@ fn priority_cookie<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, PriorityCooki
     })?;
     Ok((remaining, cookie))
 }
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn headline_level<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, (HeadlineLevel, HeadlineLevel, OrgSource<'s>)> {
+    let (remaining, stars) = is_a("*")(input)?;
+    let count = stars.len().try_into().unwrap();
+    let level = match context.get_global_settings().odd_levels_only {
+        crate::context::HeadlineLevelFilter::Odd => {
+            if count % 2 == 0 {
+                (count + 2) / 2
+            } else {
+                (count + 1) / 2
+            }
+        }
+        crate::context::HeadlineLevelFilter::OddEven => count,
+    };
+    Ok((remaining, (level, count, stars)))
+}

src/parser/horizontal_rule.rs

@@ -15,7 +15,7 @@ use crate::parser::util::start_of_line;
 use crate::types::HorizontalRule;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn horizontal_rule<'b, 'g, 'r, 's>(
+pub(crate) fn horizontal_rule<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, HorizontalRule<'s>> {

src/parser/in_buffer_settings.rs

@@ -1,28 +1,57 @@
 use nom::branch::alt;
+use nom::bytes::complete::is_not;
 use nom::bytes::complete::tag_no_case;
-use nom::character::complete::anychar;
-use nom::combinator::map;
-use nom::multi::many0;
-use nom::multi::many_till;
+use nom::bytes::complete::take_until;
+use nom::character::complete::space1;
+use nom::multi::separated_list0;
 
 use super::keyword::filtered_keyword;
 use super::keyword_todo::todo_keywords;
 use super::OrgSource;
+use crate::context::HeadlineLevelFilter;
+use crate::error::CustomError;
 use crate::error::Res;
 use crate::types::Keyword;
 use crate::GlobalSettings;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn scan_for_in_buffer_settings<'s>(
+pub(crate) fn scan_for_in_buffer_settings<'s>(
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Vec<Keyword<'s>>> {
-    // TODO: Optimization idea: since this is slicing the OrgSource at each character, it might be more efficient to do a parser that uses a search function like take_until, and wrap it in a function similar to consumed but returning the input along with the normal output, then pass all of that into a verify that confirms we were at the start of a line using the input we just returned.
+    // TODO: Write some tests to make sure this is functioning properly.
 
-    let keywords = many0(map(
-        many_till(anychar, filtered_keyword(in_buffer_settings_key)),
-        |(_, kw)| kw,
-    ))(input);
-    keywords
+    let mut keywords = Vec::new();
+    let mut remaining = input;
+    loop {
+        // Skip text until possible in_buffer_setting
+        let start_of_pound = take_until::<_, _, CustomError<_>>("#+")(remaining);
+        let start_of_pound = if let Ok((start_of_pound, _)) = start_of_pound {
+            start_of_pound
+        } else {
+            break;
+        };
+        // Go backwards to the start of the line and run the filtered_keyword parser
+        let start_of_line = start_of_pound.get_start_of_line();
+
+        let (remain, maybe_kw) = match filtered_keyword(in_buffer_settings_key)(start_of_line) {
+            Ok((remain, kw)) => (remain, Some(kw)),
+            Err(_) => {
+                let end_of_line = take_until::<_, _, CustomError<_>>("\n")(start_of_pound);
+                if let Ok((end_of_line, _)) = end_of_line {
+                    (end_of_line, None)
+                } else {
+                    break;
+                }
+            }
+        };
+
+        if let Some(kw) = maybe_kw {
+            keywords.push(kw);
+        }
+        remaining = remain;
+    }
+
+    Ok((remaining, keywords))
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -44,12 +73,13 @@ fn in_buffer_settings_key<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSou
     ))(input)
 }
 
-pub fn apply_in_buffer_settings<'g, 's, 'sf>(
+pub(crate) fn apply_in_buffer_settings<'g, 's, 'sf>(
     keywords: Vec<Keyword<'sf>>,
     original_settings: &'g GlobalSettings<'g, 's>,
 ) -> Result<GlobalSettings<'g, 's>, String> {
     let mut new_settings = original_settings.clone();
 
+    // Todo Keywords
     for kw in keywords.iter().filter(|kw| {
         kw.key.eq_ignore_ascii_case("todo")
             || kw.key.eq_ignore_ascii_case("seq_todo")
@@ -65,5 +95,51 @@ pub fn apply_in_buffer_settings<'g, 's, 'sf>(
             .extend(complete_words.into_iter().map(str::to_string));
     }
 
+    // Startup settings
+    for kw in keywords
+        .iter()
+        .filter(|kw| kw.key.eq_ignore_ascii_case("startup"))
+    {
+        let (_remaining, settings) =
+            separated_list0(space1::<&str, nom::error::Error<_>>, is_not(" \t"))(kw.value)
+                .map_err(|err: nom::Err<_>| err.to_string())?;
+        if settings.contains(&"odd") {
+            new_settings.odd_levels_only = HeadlineLevelFilter::Odd;
+        }
+        if settings.contains(&"oddeven") {
+            new_settings.odd_levels_only = HeadlineLevelFilter::OddEven;
+        }
+    }
+
     Ok(new_settings)
 }
+
+#[cfg(test)]
+mod tests {
+    use super::*;
+
+    #[test]
+    fn scan_test() -> Result<(), Box<dyn std::error::Error>> {
+        let input = OrgSource::new(
+            r#"
+foo
+   #+archive: bar
+
+baz #+category: lorem
+
+#+label: ipsum
+
+#+todo: dolar
+cat
+"#,
+        );
+        let (remaining, settings) = scan_for_in_buffer_settings(input)?;
+        assert_eq!(Into::<&str>::into(remaining), "cat\n");
+        let keys: Vec<_> = settings.iter().map(|kw| kw.key).collect();
+        // category is skipped because it is not the first non-whitespace on the line.
+        //
+        // label is skipped because it is not an in-buffer setting.
+        assert_eq!(keys, vec!["archive", "todo"]);
+        Ok(())
+    }
+}

src/parser/inline_babel_call.rs

@@ -26,7 +26,7 @@ use crate::parser::util::get_consumed;
 use crate::types::InlineBabelCall;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn inline_babel_call<'b, 'g, 'r, 's>(
+pub(crate) fn inline_babel_call<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, InlineBabelCall<'s>> {

src/parser/inline_source_block.rs

@@ -28,7 +28,7 @@ use crate::parser::util::get_consumed;
 use crate::types::InlineSourceBlock;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn inline_source_block<'b, 'g, 'r, 's>(
+pub(crate) fn inline_source_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, InlineSourceBlock<'s>> {

src/parser/keyword.rs

@@ -12,6 +12,7 @@ use nom::combinator::eof;
 use nom::combinator::not;
 use nom::combinator::peek;
 use nom::combinator::recognize;
+use nom::combinator::verify;
 use nom::multi::many_till;
 use nom::sequence::tuple;
 
@@ -23,6 +24,7 @@ use crate::error::CustomError;
 use crate::error::MyError;
 use crate::error::Res;
 use crate::parser::util::start_of_line;
+use crate::types::BabelCall;
 use crate::types::Keyword;
 
 const ORG_ELEMENT_AFFILIATED_KEYWORDS: [&'static str; 13] = [
@@ -31,7 +33,7 @@ const ORG_ELEMENT_AFFILIATED_KEYWORDS: [&'static str; 13] = [
 ];
 const ORG_ELEMENT_DUAL_KEYWORDS: [&'static str; 2] = ["caption", "results"];
 
-pub fn filtered_keyword<F: Matcher>(
+pub(crate) fn filtered_keyword<F: Matcher>(
     key_parser: F,
 ) -> impl for<'s> Fn(OrgSource<'s>) -> Res<OrgSource<'s>, Keyword<'s>> {
     move |input| _filtered_keyword(&key_parser, input)
@@ -83,7 +85,7 @@ fn _filtered_keyword<'s, F: Matcher>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn keyword<'b, 'g, 'r, 's>(
+pub(crate) fn keyword<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Keyword<'s>> {
@@ -91,7 +93,7 @@ pub fn keyword<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn affiliated_keyword<'b, 'g, 'r, 's>(
+pub(crate) fn affiliated_keyword<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Keyword<'s>> {
@@ -99,11 +101,19 @@ pub fn affiliated_keyword<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn babel_call_keyword<'b, 'g, 'r, 's>(
+pub(crate) fn babel_call_keyword<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
-) -> Res<OrgSource<'s>, Keyword<'s>> {
-    filtered_keyword(babel_call_key)(input)
+) -> Res<OrgSource<'s>, BabelCall<'s>> {
+    let (remaining, kw) = filtered_keyword(babel_call_key)(input)?;
+    Ok((
+        remaining,
+        BabelCall {
+            source: kw.source,
+            key: kw.key,
+            value: kw.value,
+        },
+    ))
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -112,11 +122,13 @@ fn babel_call_key<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>>
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn table_formula_keyword<'b, 'g, 'r, 's>(
+pub(crate) fn table_formula_keyword<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Keyword<'s>> {
-    filtered_keyword(table_formula_key)(input)
+    verify(filtered_keyword(table_formula_key), |kw| {
+        !kw.value.is_empty()
+    })(input)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]

src/parser/keyword_todo.rs

@@ -19,7 +19,7 @@ use crate::error::Res;
 /// Parses the text in the value of a #+TODO keyword.
 ///
 /// Example input: "foo bar baz | lorem ipsum"
-pub fn todo_keywords<'s>(input: &'s str) -> Res<&'s str, (Vec<&'s str>, Vec<&'s str>)> {
+pub(crate) fn todo_keywords<'s>(input: &'s str) -> Res<&'s str, (Vec<&'s str>, Vec<&'s str>)> {
     let (remaining, mut before_pipe_words) = separated_list0(space1, todo_keyword_word)(input)?;
     let (remaining, after_pipe_words) = opt(tuple((
         tuple((space0, tag("|"), space0)),
@@ -44,9 +44,17 @@ pub fn todo_keywords<'s>(input: &'s str) -> Res<&'s str, (Vec<&'s str>, Vec<&'s
 }
 
 fn todo_keyword_word<'s>(input: &'s str) -> Res<&'s str, &'s str> {
-    verify(take_till(|c| " \t\r\n|".contains(c)), |result: &str| {
+    let (remaining, keyword) = verify(take_till(|c| "( \t\r\n|".contains(c)), |result: &str| {
         !result.is_empty()
-    })(input)
+    })(input)?;
+
+    let (remaining, _) = opt(tuple((
+        tag("("),
+        take_till(|c| "() \t\r\n|".contains(c)),
+        tag(")"),
+    )))(remaining)?;
+
+    Ok((remaining, keyword))
 }
 #[cfg(test)]
 mod tests {

src/parser/latex_environment.rs

@@ -25,7 +25,7 @@ use crate::parser::util::start_of_line;
 use crate::types::LatexEnvironment;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn latex_environment<'b, 'g, 'r, 's>(
+pub(crate) fn latex_environment<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, LatexEnvironment<'s>> {

src/parser/latex_fragment.rs

@@ -24,7 +24,7 @@ use crate::parser::util::get_consumed;
 use crate::types::LatexFragment;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn latex_fragment<'b, 'g, 'r, 's>(
+pub(crate) fn latex_fragment<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, LatexFragment<'s>> {
@@ -174,7 +174,7 @@ fn dollar_char_fragment<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn pre<'b, 'g, 'r, 's>(
+fn pre<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ()> {
@@ -188,7 +188,7 @@ pub fn pre<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn post<'b, 'g, 'r, 's>(
+fn post<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ()> {
@@ -226,7 +226,7 @@ fn bordered_dollar_fragment<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn open_border<'b, 'g, 'r, 's>(
+fn open_border<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
@@ -234,7 +234,7 @@ pub fn open_border<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn close_border<'b, 'g, 'r, 's>(
+fn close_border<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ()> {

src/parser/lesser_block.rs

@@ -35,7 +35,7 @@ use crate::types::SrcBlock;
 use crate::types::VerseBlock;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn verse_block<'b, 'g, 'r, 's>(
+pub(crate) fn verse_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, VerseBlock<'s>> {
@@ -90,7 +90,7 @@ pub fn verse_block<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn comment_block<'b, 'g, 'r, 's>(
+pub(crate) fn comment_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, CommentBlock<'s>> {
@@ -130,7 +130,7 @@ pub fn comment_block<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn example_block<'b, 'g, 'r, 's>(
+pub(crate) fn example_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ExampleBlock<'s>> {
@@ -170,7 +170,7 @@ pub fn example_block<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn export_block<'b, 'g, 'r, 's>(
+pub(crate) fn export_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ExportBlock<'s>> {
@@ -211,7 +211,7 @@ pub fn export_block<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn src_block<'b, 'g, 'r, 's>(
+pub(crate) fn src_block<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, SrcBlock<'s>> {
@@ -261,11 +261,10 @@ fn data<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
     is_not("\r\n")(input)
 }
 
-fn lesser_block_end(current_name: &str) -> impl ContextMatcher {
-    let current_name_lower = current_name.to_lowercase();
-    move |context, input: OrgSource<'_>| {
-        _lesser_block_end(context, input, current_name_lower.as_str())
-    }
+fn lesser_block_end<'c>(current_name: &'c str) -> impl ContextMatcher + 'c {
+    // Since the lesser block names are statically defined in code, we can simply assert that the name is lowercase instead of causing an allocation by converting to lowercase.
+    debug_assert!(current_name == current_name.to_lowercase());
+    move |context, input: OrgSource<'_>| _lesser_block_end(context, input, current_name)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]

src/parser/line_break.rs

@@ -13,7 +13,7 @@ use crate::parser::util::get_consumed;
 use crate::types::LineBreak;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn line_break<'b, 'g, 'r, 's>(
+pub(crate) fn line_break<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, LineBreak<'s>> {

src/parser/mod.rs

@@ -37,16 +37,13 @@ mod property_drawer;
 mod radio_link;
 mod regular_link;
 mod section;
-pub mod sexp;
 mod statistics_cookie;
 mod subscript_and_superscript;
 mod table;
 mod target;
 mod text_markup;
 mod timestamp;
-mod token;
 mod util;
-pub use document::document;
 pub use document::parse;
 pub use document::parse_with_settings;
-pub use org_source::OrgSource;
+pub(crate) use org_source::OrgSource;

src/parser/object_parser.rs

@@ -32,7 +32,7 @@ use crate::parser::timestamp::timestamp;
 use crate::types::Object;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn standard_set_object<'b, 'g, 'r, 's>(
+pub(crate) fn standard_set_object<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Object<'s>> {
@@ -47,7 +47,7 @@ pub fn standard_set_object<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn minimal_set_object<'b, 'g, 'r, 's>(
+pub(crate) fn minimal_set_object<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Object<'s>> {
@@ -140,7 +140,7 @@ fn minimal_set_object_sans_plain_text<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn detect_standard_set_object_sans_plain_text<'b, 'g, 'r, 's>(
+pub(crate) fn detect_standard_set_object_sans_plain_text<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ()> {
@@ -174,7 +174,7 @@ fn detect_minimal_set_object_sans_plain_text<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn regular_link_description_set_object<'b, 'g, 'r, 's>(
+pub(crate) fn regular_link_description_set_object<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Object<'s>> {
@@ -221,7 +221,7 @@ fn regular_link_description_set_object_sans_plain_text<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn detect_regular_link_description_set_object_sans_plain_text<'b, 'g, 'r, 's>(
+fn detect_regular_link_description_set_object_sans_plain_text<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ()> {
@@ -238,7 +238,7 @@ pub fn detect_regular_link_description_set_object_sans_plain_text<'b, 'g, 'r, 's
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn table_cell_set_object<'b, 'g, 'r, 's>(
+pub(crate) fn table_cell_set_object<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Object<'s>> {
@@ -253,7 +253,7 @@ pub fn table_cell_set_object<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn table_cell_set_object_sans_plain_text<'b, 'g, 'r, 's>(
+fn table_cell_set_object_sans_plain_text<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Object<'s>> {
@@ -287,7 +287,7 @@ pub fn table_cell_set_object_sans_plain_text<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn detect_table_cell_set_object_sans_plain_text<'b, 'g, 'r, 's>(
+fn detect_table_cell_set_object_sans_plain_text<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, ()> {

src/parser/org_macro.rs

@@ -18,7 +18,7 @@ use crate::parser::util::get_consumed;
 use crate::types::OrgMacro;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn org_macro<'b, 'g, 'r, 's>(
+pub(crate) fn org_macro<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgMacro<'s>> {

src/parser/org_source.rs

@@ -1,6 +1,7 @@
 use std::ops::RangeBounds;
 
 use nom::Compare;
+use nom::FindSubstring;
 use nom::InputIter;
 use nom::InputLength;
 use nom::InputTake;
@@ -11,10 +12,10 @@ use nom::Slice;
 use crate::error::CustomError;
 use crate::error::MyError;
 
-pub type BracketDepth = i16;
+pub(crate) type BracketDepth = i16;
 
 #[derive(Copy, Clone)]
-pub struct OrgSource<'s> {
+pub(crate) struct OrgSource<'s> {
     full_source: &'s str,
     start: usize,
     end: usize, // exclusive
@@ -37,7 +38,7 @@ impl<'s> OrgSource<'s> {
     /// Returns a wrapped string that keeps track of values we need for parsing org-mode.
     ///
     /// Only call this on the full original string. Calling this on a substring can result in invalid values.
-    pub fn new(input: &'s str) -> Self {
+    pub(crate) fn new(input: &'s str) -> Self {
         OrgSource {
             full_source: input,
             start: 0,
@@ -51,37 +52,90 @@ impl<'s> OrgSource<'s> {
     }
 
     /// Get the text since the line break preceding the start of this WrappedInput.
-    pub fn text_since_line_break(&self) -> &'s str {
+    pub(crate) fn text_since_line_break(&self) -> &'s str {
         &self.full_source[self.start_of_line..self.start]
     }
 
-    pub fn len(&self) -> usize {
+    pub(crate) fn len(&self) -> usize {
         self.end - self.start
     }
 
-    pub fn get_preceding_character(&self) -> Option<char> {
+    pub(crate) fn get_byte_offset(&self) -> usize {
+        self.start
+    }
+
+    pub(crate) fn get_preceding_character(&self) -> Option<char> {
         self.preceding_character
     }
 
-    pub fn is_at_start_of_line(&self) -> bool {
+    pub(crate) fn is_at_start_of_line(&self) -> bool {
         self.start == self.start_of_line
     }
 
-    pub fn get_until(&self, other: OrgSource<'s>) -> OrgSource<'s> {
-        assert!(other.start >= self.start);
-        assert!(other.end <= self.end);
+    pub(crate) fn get_until(&self, other: OrgSource<'s>) -> OrgSource<'s> {
+        debug_assert!(other.start >= self.start);
+        debug_assert!(other.end <= self.end);
         self.slice(..(other.start - self.start))
     }
 
-    pub fn get_bracket_depth(&self) -> BracketDepth {
+    pub(crate) fn get_start_of_line(&self) -> OrgSource<'s> {
+        let skipped_text = self.text_since_line_break();
+        let mut bracket_depth = self.bracket_depth;
+        let mut brace_depth = self.brace_depth;
+        let mut parenthesis_depth = self.parenthesis_depth;
+        // Since we're going backwards, this does the opposite.
+        for byte in skipped_text.bytes() {
+            match byte {
+                b'\n' => {
+                    panic!("Should not hit a line break when only going back to the start of the line.");
+                }
+                b'[' => {
+                    bracket_depth -= 1;
+                }
+                b']' => {
+                    bracket_depth += 1;
+                }
+                b'{' => {
+                    brace_depth -= 1;
+                }
+                b'}' => {
+                    brace_depth += 1;
+                }
+                b'(' => {
+                    parenthesis_depth -= 1;
+                }
+                b')' => {
+                    parenthesis_depth += 1;
+                }
+                _ => {}
+            };
+        }
+
+        OrgSource {
+            full_source: self.full_source,
+            start: self.start_of_line,
+            end: self.end,
+            start_of_line: self.start_of_line,
+            preceding_character: if self.start_of_line > 0 {
+                Some('\n')
+            } else {
+                None
+            },
+            bracket_depth,
+            brace_depth,
+            parenthesis_depth,
+        }
+    }
+
+    pub(crate) fn get_bracket_depth(&self) -> BracketDepth {
         self.bracket_depth
     }
 
-    pub fn get_brace_depth(&self) -> BracketDepth {
+    pub(crate) fn get_brace_depth(&self) -> BracketDepth {
         self.brace_depth
     }
 
-    pub fn get_parenthesis_depth(&self) -> BracketDepth {
+    pub(crate) fn get_parenthesis_depth(&self) -> BracketDepth {
         self.parenthesis_depth
     }
 }
@@ -306,7 +360,13 @@ impl<'s> InputTakeAtPosition for OrgSource<'s> {
     }
 }
 
-pub fn convert_error<'a, I: Into<CustomError<&'a str>>>(
+impl<'n, 's> FindSubstring<&'n str> for OrgSource<'s> {
+    fn find_substring(&self, substr: &'n str) -> Option<usize> {
+        Into::<&str>::into(self).find(substr)
+    }
+}
+
+pub(crate) fn convert_error<'a, I: Into<CustomError<&'a str>>>(
     err: nom::Err<I>,
 ) -> nom::Err<CustomError<&'a str>> {
     match err {

src/parser/paragraph.rs

@@ -22,7 +22,7 @@ use crate::parser::util::start_of_line;
 use crate::types::Paragraph;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn paragraph<'b, 'g, 'r, 's>(
+pub(crate) fn paragraph<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Paragraph<'s>> {
@@ -74,7 +74,7 @@ mod tests {
     use crate::context::List;
     use crate::parser::element_parser::element;
     use crate::parser::org_source::OrgSource;
-    use crate::types::Source;
+    use crate::types::GetStandardProperties;
 
     #[test]
     fn two_paragraphs() {
@@ -87,7 +87,13 @@ mod tests {
         let (remaining, second_paragraph) =
             paragraph_matcher(remaining).expect("Parse second paragraph.");
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(first_paragraph.get_source(), "foo bar baz\n\n");
-        assert_eq!(second_paragraph.get_source(), "lorem ipsum");
+        assert_eq!(
+            first_paragraph.get_standard_properties().get_source(),
+            "foo bar baz\n\n"
+        );
+        assert_eq!(
+            second_paragraph.get_standard_properties().get_source(),
+            "lorem ipsum"
+        );
     }
 }

src/parser/plain_link.rs

@@ -5,17 +5,24 @@ use nom::character::complete::anychar;
 use nom::character::complete::none_of;
 use nom::character::complete::one_of;
 use nom::combinator::eof;
+use nom::combinator::not;
 use nom::combinator::peek;
 use nom::combinator::recognize;
 use nom::combinator::verify;
+use nom::multi::many0;
+use nom::multi::many1;
 use nom::multi::many_till;
+use nom::sequence::tuple;
 
+use super::org_source::BracketDepth;
 use super::org_source::OrgSource;
 use super::util::maybe_consume_object_trailing_whitespace_if_not_exiting;
 use crate::context::parser_with_context;
 use crate::context::ContextElement;
+use crate::context::ContextMatcher;
 use crate::context::ExitClass;
 use crate::context::ExitMatcherNode;
+use crate::context::Matcher;
 use crate::context::RefContext;
 use crate::error::CustomError;
 use crate::error::MyError;
@@ -53,7 +60,7 @@ const ORG_LINK_PARAMETERS: [&'static str; 23] = [
 ];
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn plain_link<'b, 'g, 'r, 's>(
+pub(crate) fn plain_link<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, PlainLink<'s>> {
@@ -105,7 +112,7 @@ fn post<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn protocol<'b, 'g, 'r, 's>(
+pub(crate) fn protocol<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
@@ -130,17 +137,77 @@ fn path_plain<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    // TODO: "optionally containing parenthesis-wrapped non-whitespace non-bracket substrings up to a depth of two. The string must end with either a non-punctation non-whitespace character, a forwards slash, or a parenthesis-wrapped substring"
+    let path_plain_end = path_plain_end(input.get_parenthesis_depth());
     let parser_context = ContextElement::ExitMatcherNode(ExitMatcherNode {
         class: ExitClass::Gamma,
         exit_matcher: &path_plain_end,
     });
     let parser_context = context.with_additional_node(&parser_context);
 
-    let exit_matcher = parser_with_context!(exit_matcher_parser)(&parser_context);
+    let (remaining, _components) = many1(alt((
+        parser_with_context!(path_plain_no_parenthesis)(&parser_context),
+        parser_with_context!(path_plain_parenthesis)(&parser_context),
+    )))(input)?;
 
+    let source = get_consumed(input, remaining);
+    Ok((remaining, source))
+}
+
+fn path_plain_end(starting_parenthesis_depth: BracketDepth) -> impl ContextMatcher {
+    move |context, input: OrgSource<'_>| _path_plain_end(context, input, starting_parenthesis_depth)
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn _path_plain_end<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+    starting_parenthesis_depth: BracketDepth,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    let (remaining, _leading_punctuation) = many0(verify(anychar, |c| {
+        !" \t\r\n[]<>()/".contains(*c) && c.is_ascii_punctuation()
+    }))(input)?;
+
+    let disallowed_character = recognize(one_of(" \t\r\n[]<>"))(remaining);
+    if disallowed_character.is_ok() {
+        return disallowed_character;
+    }
+
+    let current_depth = remaining.get_parenthesis_depth() - starting_parenthesis_depth;
+    if current_depth == 0 {
+        let close_parenthesis =
+            tag::<&str, OrgSource<'_>, CustomError<OrgSource<'_>>>(")")(remaining);
+        if close_parenthesis.is_ok() {
+            return close_parenthesis;
+        }
+
+        let open_parenthesis_without_match = recognize(tuple((
+            peek(tag("(")),
+            not(parser_with_context!(path_plain_parenthesis)(context)),
+        )))(remaining);
+        if open_parenthesis_without_match.is_ok() {
+            return open_parenthesis_without_match;
+        }
+    }
+
+    // many0 punctuation
+    Err(nom::Err::Error(CustomError::MyError(MyError(
+        "No path plain end".into(),
+    ))))
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn path_plain_no_parenthesis<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
     let (remaining, path) = recognize(verify(
-        many_till(anychar, peek(exit_matcher)),
+        many_till(
+            anychar,
+            alt((
+                peek(path_plain_no_parenthesis_disallowed_character),
+                parser_with_context!(exit_matcher_parser)(context),
+            )),
+        ),
         |(children, _exit_contents)| !children.is_empty(),
     ))(input)?;
 
@@ -148,14 +215,65 @@ fn path_plain<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn path_plain_end<'b, 'g, 'r, 's>(
-    _context: RefContext<'b, 'g, 'r, 's>,
+fn path_plain_no_parenthesis_disallowed_character<'s>(
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    recognize(many_till(
-        verify(anychar, |c| {
-            *c != '/' && (c.is_ascii_punctuation() || c.is_whitespace())
-        }),
-        one_of(" \t\r\n()[]<>"),
-    ))(input)
+    recognize(verify(anychar, |c| {
+        c.is_whitespace() || "()[]<>".contains(*c)
+    }))(input)
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn path_plain_parenthesis<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    let (remaining, _opening) = tag("(")(input)?;
+    let starting_depth = remaining.get_parenthesis_depth();
+
+    let (remaining, _path) = recognize(verify(
+        many_till(
+            anychar,
+            alt((
+                peek(path_plain_parenthesis_end(starting_depth)),
+                parser_with_context!(exit_matcher_parser)(context),
+            )),
+        ),
+        |(children, _exit_contents)| !children.is_empty(),
+    ))(remaining)?;
+    let (remaining, _opening) = tag(")")(remaining)?;
+    let source = get_consumed(input, remaining);
+
+    Ok((remaining, source))
+}
+
+fn path_plain_parenthesis_end(starting_parenthesis_depth: BracketDepth) -> impl Matcher {
+    move |input: OrgSource<'_>| _path_plain_parenthesis_end(input, starting_parenthesis_depth)
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn _path_plain_parenthesis_end<'s>(
+    input: OrgSource<'s>,
+    starting_parenthesis_depth: BracketDepth,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    let current_depth = input.get_parenthesis_depth() - starting_parenthesis_depth;
+    if current_depth < 0 {
+        // This shouldn't be possible because if depth is 0 then a closing parenthesis should end the link.
+        unreachable!("Exceeded plain link parenthesis depth.")
+    }
+    if current_depth == 0 {
+        let close_parenthesis = tag::<&str, OrgSource<'_>, CustomError<OrgSource<'_>>>(")")(input);
+        if close_parenthesis.is_ok() {
+            return close_parenthesis;
+        }
+    }
+    if current_depth == 1 {
+        let open_parenthesis = tag::<&str, OrgSource<'_>, CustomError<OrgSource<'_>>>("(")(input);
+        if open_parenthesis.is_ok() {
+            return open_parenthesis;
+        }
+    }
+    Err(nom::Err::Error(CustomError::MyError(MyError(
+        "No closing parenthesis".into(),
+    ))))
 }

src/parser/plain_list.rs

@@ -1,11 +1,13 @@
 use nom::branch::alt;
 use nom::bytes::complete::tag;
+use nom::character::complete::anychar;
 use nom::character::complete::digit1;
 use nom::character::complete::line_ending;
 use nom::character::complete::one_of;
 use nom::character::complete::space0;
 use nom::character::complete::space1;
 use nom::combinator::eof;
+use nom::combinator::map;
 use nom::combinator::not;
 use nom::combinator::opt;
 use nom::combinator::peek;
@@ -20,6 +22,7 @@ use super::element_parser::element;
 use super::object_parser::standard_set_object;
 use super::org_source::OrgSource;
 use super::util::include_input;
+use super::util::indentation_level;
 use super::util::non_whitespace_character;
 use crate::context::parser_with_context;
 use crate::context::ContextElement;
@@ -34,21 +37,27 @@ use crate::parser::util::blank_line;
 use crate::parser::util::exit_matcher_parser;
 use crate::parser::util::get_consumed;
 use crate::parser::util::maybe_consume_trailing_whitespace_if_not_exiting;
+use crate::parser::util::org_space;
 use crate::parser::util::start_of_line;
+use crate::types::CheckboxType;
+use crate::types::IndentationLevel;
 use crate::types::Object;
 use crate::types::PlainList;
 use crate::types::PlainListItem;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn detect_plain_list<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+pub(crate) fn detect_plain_list<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, ()> {
     if verify(
         tuple((
             start_of_line,
             space0,
-            bullet,
+            parser_with_context!(bullet)(context),
             alt((space1, line_ending, eof)),
         )),
-        |(_start, indent, bull, _after_whitespace)| {
+        |(_start, indent, (_bullet_type, bull), _after_whitespace)| {
             Into::<&str>::into(bull) != "*" || indent.len() > 0
         },
     )(input)
@@ -62,7 +71,7 @@ pub fn detect_plain_list<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn plain_list<'b, 'g, 'r, 's>(
+pub(crate) fn plain_list<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, PlainList<'s>> {
@@ -80,7 +89,7 @@ pub fn plain_list<'b, 'g, 'r, 's>(
     let parser_context = parser_context.with_additional_node(&contexts[2]);
     // children stores tuple of (input string, parsed object) so we can re-parse the final item
     let mut children = Vec::new();
-    let mut first_item_indentation: Option<usize> = None;
+    let mut first_item_indentation: Option<IndentationLevel> = None;
     let mut remaining = input;
 
     // The final list item does not consume trailing blank lines (which instead get consumed by the list). We have three options here:
@@ -136,49 +145,32 @@ pub fn plain_list<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn plain_list_item<'b, 'g, 'r, 's>(
+fn plain_list_item<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, PlainListItem<'s>> {
     start_of_line(input)?;
-    let (remaining, leading_whitespace) = space0(input)?;
-    // It is fine that we get the indent level using the number of bytes rather than the number of characters because nom's space0 only matches space and tab (0x20 and 0x09)
-    let indent_level = leading_whitespace.len();
-    let (remaining, bull) = verify(bullet, |bull: &OrgSource<'_>| {
-        Into::<&str>::into(bull) != "*" || indent_level > 0
-    })(remaining)?;
+    let (remaining, (indent_level, _leading_whitespace)) = indentation_level(context, input)?;
+    let (remaining, (bullet_type, bull)) = verify(
+        parser_with_context!(bullet)(context),
+        |(_bullet_type, bull)| Into::<&str>::into(bull) != "*" || indent_level > 0,
+    )(remaining)?;
 
-    let (remaining, _maybe_counter_set) =
-        opt(tuple((space1, tag("[@"), counter, tag("]"))))(remaining)?;
+    let (remaining, _maybe_counter_set) = opt(tuple((
+        space1,
+        tag("[@"),
+        parser_with_context!(counter)(context),
+        tag("]"),
+    )))(remaining)?;
 
-    // TODO: parse checkbox
+    let (remaining, maybe_checkbox) = opt(tuple((space1, item_checkbox)))(remaining)?;
 
-    let (remaining, maybe_tag) =
-        opt(tuple((space1, parser_with_context!(item_tag)(context))))(remaining)?;
-
-    let maybe_contentless_item: Res<OrgSource<'_>, ()> = peek(parser_with_context!(
-        detect_contentless_item_contents
-    )(context))(remaining);
-    match maybe_contentless_item {
-        Ok((_rem, _ws)) => {
-            let (remaining, _trailing_ws) = opt(blank_line)(remaining)?;
-            let source = get_consumed(input, remaining);
-            return Ok((
-                remaining,
-                PlainListItem {
-                    source: source.into(),
-                    indentation: indent_level,
-                    bullet: bull.into(),
-                    tag: maybe_tag
-                        .map(|(_ws, item_tag)| item_tag)
-                        .unwrap_or(Vec::new()),
-                    children: Vec::new(),
-                },
-            ));
-        }
-        Err(_) => {}
+    let (remaining, maybe_tag) = if let BulletType::Unordered = bullet_type {
+        opt(tuple((space1, parser_with_context!(item_tag)(context))))(remaining)?
+    } else {
+        (remaining, None)
     };
-    let (remaining, _ws) = item_tag_post_gap(context, remaining)?;
+
     let exit_matcher = plain_list_item_end(indent_level);
     let contexts = [
         ContextElement::ConsumeTrailingWhitespace(true),
@@ -190,6 +182,35 @@ pub fn plain_list_item<'b, 'g, 'r, 's>(
     let parser_context = context.with_additional_node(&contexts[0]);
     let parser_context = parser_context.with_additional_node(&contexts[1]);
 
+    let maybe_contentless_item: Res<OrgSource<'_>, ()> = peek(parser_with_context!(
+        detect_contentless_item_contents
+    )(&parser_context))(remaining);
+    match maybe_contentless_item {
+        Ok((_rem, _ws)) => {
+            let (remaining, _trailing_ws) = if context.should_consume_trailing_whitespace() {
+                recognize(alt((recognize(many1(blank_line)), eof)))(remaining)?
+            } else {
+                recognize(alt((blank_line, eof)))(remaining)?
+            };
+            let source = get_consumed(input, remaining);
+            return Ok((
+                remaining,
+                PlainListItem {
+                    source: source.into(),
+                    indentation: indent_level,
+                    bullet: bull.into(),
+                    checkbox: None,
+                    tag: maybe_tag
+                        .map(|(_ws, item_tag)| item_tag)
+                        .unwrap_or(Vec::new()),
+                    children: Vec::new(),
+                },
+            ));
+        }
+        Err(_) => {}
+    };
+    let (remaining, _ws) = item_tag_post_gap(&parser_context, remaining)?;
+
     let (mut remaining, (mut children, _exit_contents)) = many_till(
         include_input(parser_with_context!(element(true))(&parser_context)),
         parser_with_context!(exit_matcher_parser)(&parser_context),
@@ -218,6 +239,8 @@ pub fn plain_list_item<'b, 'g, 'r, 's>(
             source: source.into(),
             indentation: indent_level,
             bullet: bull.into(),
+            checkbox: maybe_checkbox
+                .map(|(_, (checkbox_type, source))| (checkbox_type, Into::<&str>::into(source))),
             tag: maybe_tag
                 .map(|(_ws, item_tag)| item_tag)
                 .unwrap_or(Vec::new()),
@@ -226,19 +249,46 @@ pub fn plain_list_item<'b, 'g, 'r, 's>(
     ));
 }
 
-#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn bullet<'s>(i: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    alt((
-        tag("*"),
-        tag("-"),
-        tag("+"),
-        recognize(tuple((counter, alt((tag("."), tag(")")))))),
-    ))(i)
+#[derive(Debug)]
+enum BulletType {
+    Ordered,
+    Unordered,
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn counter<'s>(i: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    alt((recognize(one_of("abcdefghijklmnopqrstuvwxyz")), digit1))(i)
+fn bullet<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, (BulletType, OrgSource<'s>)> {
+    alt((
+        map(tag("*"), |bull| (BulletType::Unordered, bull)),
+        map(tag("-"), |bull| (BulletType::Unordered, bull)),
+        map(tag("+"), |bull| (BulletType::Unordered, bull)),
+        map(
+            recognize(tuple((
+                parser_with_context!(counter)(context),
+                alt((tag("."), tag(")"))),
+            ))),
+            |bull| (BulletType::Ordered, bull),
+        ),
+    ))(input)
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn counter<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    if context.get_global_settings().org_list_allow_alphabetical {
+        alt((
+            recognize(one_of(
+                "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ",
+            )),
+            digit1,
+        ))(input)
+    } else {
+        digit1(input)
+    }
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -254,7 +304,7 @@ fn plain_list_end<'b, 'g, 'r, 's>(
     )))(input)
 }
 
-const fn plain_list_item_end(indent_level: usize) -> impl ContextMatcher {
+const fn plain_list_item_end(indent_level: IndentationLevel) -> impl ContextMatcher {
     let line_indented_lte_matcher = line_indented_lte(indent_level);
     move |context, input: OrgSource<'_>| {
         _plain_list_item_end(context, input, &line_indented_lte_matcher)
@@ -277,20 +327,23 @@ fn _plain_list_item_end<'b, 'g, 'r, 's>(
     )))(input)
 }
 
-const fn line_indented_lte(indent_level: usize) -> impl ContextMatcher {
+const fn line_indented_lte(indent_level: IndentationLevel) -> impl ContextMatcher {
     move |context, input: OrgSource<'_>| _line_indented_lte(context, input, indent_level)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
 fn _line_indented_lte<'b, 'g, 'r, 's>(
-    _context: RefContext<'b, 'g, 'r, 's>,
+    context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
-    indent_level: usize,
+    indent_level: IndentationLevel,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
     let matched = recognize(verify(
-        tuple((space0::<OrgSource<'_>, _>, non_whitespace_character)),
+        tuple((
+            parser_with_context!(indentation_level)(context),
+            non_whitespace_character,
+        )),
         // It is fine that we get the indent level using the number of bytes rather than the number of characters because nom's space0 only matches space and tab (0x20 and 0x09)
-        |(_space0, _anychar)| _space0.len() <= indent_level,
+        |((indentation_level, _leading_whitespace), _anychar)| *indentation_level <= indent_level,
     ))(input)?;
 
     Ok(matched)
@@ -301,18 +354,11 @@ fn item_tag<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Vec<Object<'s>>> {
-    let contexts = [
-        ContextElement::ExitMatcherNode(ExitMatcherNode {
-            class: ExitClass::Gamma,
-            exit_matcher: &item_tag_line_ending_end,
-        }),
-        ContextElement::ExitMatcherNode(ExitMatcherNode {
-            class: ExitClass::Delta,
-            exit_matcher: &item_tag_end,
-        }),
-    ];
-    let parser_context = context.with_additional_node(&contexts[0]);
-    let parser_context = parser_context.with_additional_node(&contexts[1]);
+    let parser_context = ContextElement::ExitMatcherNode(ExitMatcherNode {
+        class: ExitClass::Gamma,
+        exit_matcher: &item_tag_end,
+    });
+    let parser_context = context.with_additional_node(&parser_context);
     let (remaining, (children, _exit_contents)) = verify(
         many_till(
             // TODO: Should this be using a different set like the minimal set?
@@ -321,7 +367,7 @@ fn item_tag<'b, 'g, 'r, 's>(
         ),
         |(children, _exit_contents)| !children.is_empty(),
     )(input)?;
-    let (remaining, _) = tuple((one_of(" \t"), tag("::")))(remaining)?;
+    let (remaining, _) = item_tag_divider(remaining)?;
     Ok((remaining, children))
 }
 
@@ -330,19 +376,22 @@ fn item_tag_end<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    recognize(tuple((
-        one_of(" \t"),
-        tag("::"),
-        alt((recognize(one_of(" \t")), line_ending, eof)),
-    )))(input)
+    alt((item_tag_divider, line_ending))(input)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn item_tag_line_ending_end<'b, 'g, 'r, 's>(
-    _context: RefContext<'b, 'g, 'r, 's>,
-    input: OrgSource<'s>,
-) -> Res<OrgSource<'s>, OrgSource<'s>> {
-    line_ending(input)
+fn item_tag_divider<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    recognize(tuple((
+        one_of(" \t"),
+        tag("::"),
+        peek(tuple((
+            opt(tuple((
+                peek(one_of(" \t")),
+                many_till(anychar, peek(alt((item_tag_divider, line_ending, eof)))),
+            ))),
+            alt((line_ending, eof)),
+        ))),
+    )))(input)
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
@@ -366,6 +415,18 @@ fn item_tag_post_gap<'b, 'g, 'r, 's>(
     )(input)
 }
 
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn item_checkbox<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, (CheckboxType, OrgSource<'s>)> {
+    alt((
+        map(
+            recognize(tuple((tag("["), org_space, tag("]")))),
+            |capture| (CheckboxType::Off, capture),
+        ),
+        map(tag("[-]"), |capture| (CheckboxType::Trans, capture)),
+        map(tag("[X]"), |capture| (CheckboxType::On, capture)),
+    ))(input)
+}
+
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
 fn detect_contentless_item_contents<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
@@ -384,7 +445,7 @@ mod tests {
     use crate::context::Context;
     use crate::context::GlobalSettings;
     use crate::context::List;
-    use crate::types::Source;
+    use crate::types::GetStandardProperties;
 
     #[test]
     fn plain_list_item_empty() {
@@ -395,7 +456,7 @@ mod tests {
         let plain_list_item_matcher = parser_with_context!(plain_list_item)(&initial_context);
         let (remaining, result) = plain_list_item_matcher(input).unwrap();
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(result.source, "1.");
+        assert_eq!(result.get_standard_properties().get_source(), "1.");
     }
 
     #[test]
@@ -407,7 +468,7 @@ mod tests {
         let plain_list_item_matcher = parser_with_context!(plain_list_item)(&initial_context);
         let (remaining, result) = plain_list_item_matcher(input).unwrap();
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(result.source, "1. foo");
+        assert_eq!(result.get_standard_properties().get_source(), "1. foo");
     }
 
     #[test]
@@ -419,7 +480,7 @@ mod tests {
         let plain_list_matcher = parser_with_context!(plain_list)(&initial_context);
         let (remaining, result) = plain_list_matcher(input).unwrap();
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(result.source, "1.");
+        assert_eq!(result.get_standard_properties().get_source(), "1.");
     }
 
     #[test]
@@ -431,7 +492,7 @@ mod tests {
         let plain_list_matcher = parser_with_context!(plain_list)(&initial_context);
         let (remaining, result) = plain_list_matcher(input).unwrap();
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(result.source, "1. foo");
+        assert_eq!(result.get_standard_properties().get_source(), "1. foo");
     }
 
     #[test]
@@ -478,7 +539,7 @@ mod tests {
             plain_list_matcher(input).expect("Should parse the plain list successfully.");
         assert_eq!(Into::<&str>::into(remaining), "   ipsum\n");
         assert_eq!(
-            result.get_source(),
+            result.get_standard_properties().get_source(),
             r#"1. foo
 2. bar
    baz
@@ -506,7 +567,7 @@ baz"#,
             plain_list_matcher(input).expect("Should parse the plain list successfully.");
         assert_eq!(Into::<&str>::into(remaining), "baz");
         assert_eq!(
-            result.get_source(),
+            result.get_standard_properties().get_source(),
             r#"1. foo
    1. bar
 
@@ -539,7 +600,7 @@ dolar"#,
             plain_list_matcher(input).expect("Should parse the plain list successfully.");
         assert_eq!(Into::<&str>::into(remaining), "dolar");
         assert_eq!(
-            result.get_source(),
+            result.get_standard_properties().get_source(),
             r#"1. foo
 
    bar
@@ -561,21 +622,30 @@ dolar"#,
             r#"+
 "#,
         );
-        let result = detect_plain_list(input);
+        let global_settings = GlobalSettings::default();
+        let initial_context = ContextElement::document_context();
+        let initial_context = Context::new(&global_settings, List::new(&initial_context));
+        let result = detect_plain_list(&initial_context, input);
         assert!(result.is_ok());
     }
 
     #[test]
     fn detect_eof() {
         let input = OrgSource::new(r#"+"#);
-        let result = detect_plain_list(input);
+        let global_settings = GlobalSettings::default();
+        let initial_context = ContextElement::document_context();
+        let initial_context = Context::new(&global_settings, List::new(&initial_context));
+        let result = detect_plain_list(&initial_context, input);
         assert!(result.is_ok());
     }
 
     #[test]
     fn detect_no_gap() {
         let input = OrgSource::new(r#"+foo"#);
-        let result = detect_plain_list(input);
+        let global_settings = GlobalSettings::default();
+        let initial_context = ContextElement::document_context();
+        let initial_context = Context::new(&global_settings, List::new(&initial_context));
+        let result = detect_plain_list(&initial_context, input);
         // Since there is no whitespace after the '+' this is a paragraph, not a plain list.
         assert!(result.is_err());
     }
@@ -583,7 +653,10 @@ dolar"#,
     #[test]
     fn detect_with_gap() {
         let input = OrgSource::new(r#"+ foo"#);
-        let result = detect_plain_list(input);
+        let global_settings = GlobalSettings::default();
+        let initial_context = ContextElement::document_context();
+        let initial_context = Context::new(&global_settings, List::new(&initial_context));
+        let result = detect_plain_list(&initial_context, input);
         assert!(result.is_ok());
     }
 }

src/parser/plain_text.rs

@@ -1,22 +1,31 @@
 use nom::branch::alt;
-use nom::bytes::complete::tag;
+use nom::bytes::complete::is_not;
+use nom::bytes::complete::tag_no_case;
 use nom::character::complete::anychar;
-use nom::combinator::map;
+use nom::character::complete::line_ending;
+use nom::character::complete::one_of;
+use nom::combinator::eof;
 use nom::combinator::peek;
 use nom::combinator::recognize;
 use nom::combinator::verify;
+use nom::multi::many1;
 use nom::multi::many_till;
+use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
 use super::radio_link::RematchObject;
 use super::util::exit_matcher_parser;
+use super::util::get_consumed;
+use super::util::org_space_or_line_ending;
 use crate::context::parser_with_context;
 use crate::context::RefContext;
+use crate::error::CustomError;
+use crate::error::MyError;
 use crate::error::Res;
 use crate::types::Object;
 use crate::types::PlainText;
 
-pub fn plain_text<F>(
+pub(crate) fn plain_text<F>(
     end_condition: F,
 ) -> impl for<'b, 'g, 'r, 's> Fn(
     RefContext<'b, 'g, 'r, 's>,
@@ -72,11 +81,58 @@ impl<'x> RematchObject<'x> for PlainText<'x> {
         _context: RefContext<'b, 'g, 'r, 's>,
         input: OrgSource<'s>,
     ) -> Res<OrgSource<'s>, Object<'s>> {
-        map(tag(self.source), |s| {
+        let mut remaining = input;
+        let mut goal = self.source;
+
+        loop {
+            if goal.is_empty() {
+                break;
+            }
+
+            let is_not_whitespace = is_not::<&str, &str, CustomError<_>>(" \t\r\n")(goal);
+            match is_not_whitespace {
+                Ok((new_goal, payload)) => {
+                    let (new_remaining, _) = tuple((
+                        tag_no_case(payload),
+                        // TODO: Test to see what the REAL condition is. Checking for not-alphabetic works fine for now, but the real criteria might be something like the plain text exit matcher.
+                        peek(alt((
+                            recognize(verify(anychar, |c| !c.is_alphanumeric())),
+                            eof,
+                        ))),
+                    ))(remaining)?;
+                    remaining = new_remaining;
+                    goal = new_goal;
+                    continue;
+                }
+                Err(_) => {}
+            };
+
+            let is_whitespace = recognize(many1(alt((
+                recognize(one_of::<&str, &str, CustomError<_>>(" \t")),
+                line_ending,
+            ))))(goal);
+            match is_whitespace {
+                Ok((new_goal, _)) => {
+                    let (new_remaining, _) = many1(org_space_or_line_ending)(remaining)?;
+                    remaining = new_remaining;
+                    goal = new_goal;
+                    continue;
+                }
+                Err(_) => {}
+            };
+
+            return Err(nom::Err::Error(CustomError::MyError(MyError(
+                "Target does not match.".into(),
+            ))));
+        }
+
+        let source = get_consumed(input, remaining);
+        Ok((
+            remaining,
             Object::PlainText(PlainText {
-                source: Into::<&str>::into(s),
-            })
-        })(input)
+                source: Into::<&str>::into(source),
+            }),
+        ))
     }
 }
 
@@ -90,7 +146,7 @@ mod tests {
     use crate::context::GlobalSettings;
     use crate::context::List;
     use crate::parser::object_parser::detect_standard_set_object_sans_plain_text;
-    use crate::types::Source;
+    use crate::types::GetStandardProperties;
 
     #[test]
     fn plain_text_simple() {
@@ -103,6 +159,9 @@ mod tests {
         ))(&initial_context);
         let (remaining, result) = map(plain_text_matcher, Object::PlainText)(input).unwrap();
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(result.get_source(), Into::<&str>::into(input));
+        assert_eq!(
+            result.get_standard_properties().get_source(),
+            Into::<&str>::into(input)
+        );
     }
 }

src/parser/planning.rs

@@ -1,16 +1,16 @@
 use nom::branch::alt;
-use nom::bytes::complete::is_not;
 use nom::bytes::complete::tag;
 use nom::bytes::complete::tag_no_case;
-use nom::character::complete::line_ending;
 use nom::character::complete::space0;
 use nom::character::complete::space1;
-use nom::combinator::eof;
-use nom::multi::separated_list1;
+use nom::multi::many1;
 use nom::sequence::tuple;
 
 use super::org_source::OrgSource;
+use super::timestamp::timestamp;
 use super::util::maybe_consume_trailing_whitespace_if_not_exiting;
+use super::util::org_line_ending;
+use crate::context::parser_with_context;
 use crate::context::RefContext;
 use crate::error::Res;
 use crate::parser::util::get_consumed;
@@ -18,14 +18,15 @@ use crate::parser::util::start_of_line;
 use crate::types::Planning;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn planning<'b, 'g, 'r, 's>(
+pub(crate) fn planning<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Planning<'s>> {
     start_of_line(input)?;
     let (remaining, _leading_whitespace) = space0(input)?;
-    let (remaining, _planning_parameters) = separated_list1(space1, planning_parameter)(remaining)?;
-    let (remaining, _trailing_ws) = tuple((space0, alt((line_ending, eof))))(remaining)?;
+    let (remaining, _planning_parameters) =
+        many1(parser_with_context!(planning_parameter)(context))(remaining)?;
+    let (remaining, _trailing_ws) = tuple((space0, org_line_ending))(remaining)?;
 
     let (remaining, _trailing_ws) =
         maybe_consume_trailing_whitespace_if_not_exiting(context, remaining)?;
@@ -40,15 +41,17 @@ pub fn planning<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-fn planning_parameter<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, OrgSource<'s>> {
+fn planning_parameter<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
     let (remaining, _planning_type) = alt((
         tag_no_case("DEADLINE"),
         tag_no_case("SCHEDULED"),
         tag_no_case("CLOSED"),
     ))(input)?;
     let (remaining, _gap) = tuple((tag(":"), space1))(remaining)?;
-    // TODO: Make this invoke the real timestamp parser.
-    let (remaining, _timestamp) = tuple((tag("<"), is_not("\r\n>"), tag(">")))(remaining)?;
+    let (remaining, _timestamp) = timestamp(context, remaining)?;
     let source = get_consumed(input, remaining);
     Ok((remaining, source))
 }

src/parser/property_drawer.rs

@@ -31,7 +31,7 @@ use crate::types::NodeProperty;
 use crate::types::PropertyDrawer;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn property_drawer<'b, 'g, 'r, 's>(
+pub(crate) fn property_drawer<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, PropertyDrawer<'s>> {

src/parser/radio_link.rs

@@ -23,7 +23,7 @@ use crate::types::RadioLink;
 use crate::types::RadioTarget;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn radio_link<'b, 'g, 'r, 's>(
+pub(crate) fn radio_link<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, RadioLink<'s>> {
@@ -47,7 +47,7 @@ pub fn radio_link<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn rematch_target<'x, 'b, 'g, 'r, 's>(
+pub(crate) fn rematch_target<'x, 'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     target: &'x Vec<Object<'x>>,
     input: OrgSource<'s>,
@@ -62,6 +62,22 @@ pub fn rematch_target<'x, 'b, 'g, 'r, 's>(
                 remaining = new_remaining;
                 new_matches.push(new_match);
             }
+            Object::Italic(italic) => {
+                let (new_remaining, new_match) = italic.rematch_object(context, remaining)?;
+                remaining = new_remaining;
+                new_matches.push(new_match);
+            }
+            Object::Underline(underline) => {
+                let (new_remaining, new_match) = underline.rematch_object(context, remaining)?;
+                remaining = new_remaining;
+                new_matches.push(new_match);
+            }
+            Object::StrikeThrough(strikethrough) => {
+                let (new_remaining, new_match) =
+                    strikethrough.rematch_object(context, remaining)?;
+                remaining = new_remaining;
+                new_matches.push(new_match);
+            }
             Object::PlainText(plaintext) => {
                 let (new_remaining, new_match) = plaintext.rematch_object(context, remaining)?;
                 remaining = new_remaining;
@@ -78,7 +94,7 @@ pub fn rematch_target<'x, 'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn radio_target<'b, 'g, 'r, 's>(
+pub(crate) fn radio_target<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, RadioTarget<'s>> {
@@ -118,7 +134,7 @@ fn radio_target_end<'b, 'g, 'r, 's>(
     alt((tag("<"), tag(">"), line_ending))(input)
 }
 
-pub trait RematchObject<'x> {
+pub(crate) trait RematchObject<'x> {
     fn rematch_object<'b, 'g, 'r, 's>(
         &'x self,
         context: RefContext<'b, 'g, 'r, 's>,
@@ -135,8 +151,8 @@ mod tests {
     use crate::parser::element_parser::element;
     use crate::types::Bold;
     use crate::types::Element;
+    use crate::types::GetStandardProperties;
     use crate::types::PlainText;
-    use crate::types::Source;
 
     #[test]
     fn plain_text_radio_target() {
@@ -156,7 +172,10 @@ mod tests {
             _ => panic!("Should be a paragraph!"),
         };
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(first_paragraph.get_source(), "foo bar baz");
+        assert_eq!(
+            first_paragraph.get_standard_properties().get_source(),
+            "foo bar baz"
+        );
         assert_eq!(first_paragraph.children.len(), 3);
         assert_eq!(
             first_paragraph
@@ -192,7 +211,10 @@ mod tests {
             _ => panic!("Should be a paragraph!"),
         };
         assert_eq!(Into::<&str>::into(remaining), "");
-        assert_eq!(first_paragraph.get_source(), "foo *bar* baz");
+        assert_eq!(
+            first_paragraph.get_standard_properties().get_source(),
+            "foo *bar* baz"
+        );
         assert_eq!(first_paragraph.children.len(), 3);
         assert_eq!(
             first_paragraph

src/parser/regular_link.rs

@@ -2,7 +2,7 @@ use nom::branch::alt;
 use nom::bytes::complete::escaped;
 use nom::bytes::complete::tag;
 use nom::bytes::complete::take_till1;
-use nom::character::complete::one_of;
+use nom::character::complete::anychar;
 use nom::combinator::verify;
 use nom::multi::many_till;
 
@@ -21,7 +21,7 @@ use crate::types::Object;
 use crate::types::RegularLink;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn regular_link<'b, 'g, 'r, 's>(
+pub(crate) fn regular_link<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, RegularLink<'s>> {
@@ -32,7 +32,7 @@ pub fn regular_link<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn regular_link_without_description<'b, 'g, 'r, 's>(
+fn regular_link_without_description<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, RegularLink<'s>> {
@@ -51,7 +51,7 @@ pub fn regular_link_without_description<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn regular_link_with_description<'b, 'g, 'r, 's>(
+fn regular_link_with_description<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, RegularLink<'s>> {
@@ -72,23 +72,23 @@ pub fn regular_link_with_description<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn pathreg<'b, 'g, 'r, 's>(
+fn pathreg<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, OrgSource<'s>> {
     let (remaining, path) = escaped(
         take_till1(|c| match c {
-            '\\' | ']' => true,
+            '\\' | '[' | ']' => true,
             _ => false,
         }),
         '\\',
-        one_of(r#"]"#),
+        anychar,
     )(input)?;
     Ok((remaining, path))
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn description<'b, 'g, 'r, 's>(
+fn description<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Vec<Object<'s>>> {

src/parser/section.rs

@@ -25,7 +25,7 @@ use crate::types::Element;
 use crate::types::Section;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn zeroth_section<'b, 'g, 'r, 's>(
+pub(crate) fn zeroth_section<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Section<'s>> {
@@ -83,7 +83,7 @@ pub fn zeroth_section<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn section<'b, 'g, 'r, 's>(
+pub(crate) fn section<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     mut input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Section<'s>> {

src/parser/statistics_cookie.rs

@@ -13,7 +13,7 @@ use crate::error::Res;
 use crate::types::StatisticsCookie;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn statistics_cookie<'b, 'g, 'r, 's>(
+pub(crate) fn statistics_cookie<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, StatisticsCookie<'s>> {
@@ -24,7 +24,7 @@ pub fn statistics_cookie<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn percent_statistics_cookie<'b, 'g, 'r, 's>(
+fn percent_statistics_cookie<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, StatisticsCookie<'s>> {
@@ -45,7 +45,7 @@ pub fn percent_statistics_cookie<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn fraction_statistics_cookie<'b, 'g, 'r, 's>(
+fn fraction_statistics_cookie<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, StatisticsCookie<'s>> {

src/parser/subscript_and_superscript.rs

@@ -23,6 +23,7 @@ use crate::context::ContextElement;
 use crate::context::ContextMatcher;
 use crate::context::ExitClass;
 use crate::context::ExitMatcherNode;
+use crate::context::Matcher;
 use crate::context::RefContext;
 use crate::error::CustomError;
 use crate::error::MyError;
@@ -33,7 +34,7 @@ use crate::types::Subscript;
 use crate::types::Superscript;
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn detect_subscript_or_superscript<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+pub(crate) fn detect_subscript_or_superscript<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
     // This does not have to detect all valid subscript/superscript but all that it detects must be valid.
     let (remaining, _) = one_of("_^")(input)?;
     pre(input)?;
@@ -46,7 +47,7 @@ pub fn detect_subscript_or_superscript<'s>(input: OrgSource<'s>) -> Res<OrgSourc
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn subscript<'b, 'g, 'r, 's>(
+pub(crate) fn subscript<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Subscript<'s>> {
@@ -66,7 +67,7 @@ pub fn subscript<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn superscript<'b, 'g, 'r, 's>(
+pub(crate) fn superscript<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Superscript<'s>> {
@@ -112,6 +113,10 @@ fn script_body<'b, 'g, 'r, 's>(
         map(parser_with_context!(script_with_braces)(context), |body| {
             ScriptBody::WithBraces(body.into())
         }),
+        map(
+            parser_with_context!(script_with_parenthesis)(context),
+            |body| ScriptBody::Braceless(body.into()),
+        ),
     ))(input)
 }
 
@@ -199,3 +204,49 @@ fn _script_with_braces_end<'b, 'g, 'r, 's>(
     }
     tag("}")(input)
 }
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn script_with_parenthesis<'b, 'g, 'r, 's>(
+    context: RefContext<'b, 'g, 'r, 's>,
+    input: OrgSource<'s>,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    let (remaining, _) = tag("(")(input)?;
+    let exit_with_depth = script_with_parenthesis_end(remaining.get_parenthesis_depth());
+
+    let (remaining, _) = many_till(
+        anychar,
+        alt((
+            peek(exit_with_depth),
+            parser_with_context!(exit_matcher_parser)(context),
+        )),
+    )(remaining)?;
+
+    let (remaining, _) = tag(")")(remaining)?;
+    let source = get_consumed(input, remaining);
+    Ok((remaining, source))
+}
+
+fn script_with_parenthesis_end(starting_parenthesis_depth: BracketDepth) -> impl Matcher {
+    move |input: OrgSource<'_>| _script_with_parenthesis_end(input, starting_parenthesis_depth)
+}
+
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+fn _script_with_parenthesis_end<'s>(
+    input: OrgSource<'s>,
+    starting_parenthesis_depth: BracketDepth,
+) -> Res<OrgSource<'s>, OrgSource<'s>> {
+    let current_depth = input.get_parenthesis_depth() - starting_parenthesis_depth;
+    if current_depth < 0 {
+        // This shouldn't be possible because if depth is 0 then a closing bracket should end the citation.
+        unreachable!("Exceeded citation key suffix bracket depth.")
+    }
+    if current_depth == 0 {
+        let close_parenthesis = tag::<&str, OrgSource<'_>, CustomError<OrgSource<'_>>>(")")(input);
+        if close_parenthesis.is_ok() {
+            return close_parenthesis;
+        }
+    }
+    Err(nom::Err::Error(CustomError::MyError(MyError(
+        "No script parenthesis end.".into(),
+    ))))
+}

src/parser/table.rs

@@ -33,7 +33,7 @@ use crate::types::TableRow;
 ///
 /// This is not the table.el style.
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn org_mode_table<'b, 'g, 'r, 's>(
+pub(crate) fn org_mode_table<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, Table<'s>> {
@@ -61,7 +61,6 @@ pub fn org_mode_table<'b, 'g, 'r, 's>(
     let (remaining, formulas) =
         many0(parser_with_context!(table_formula_keyword)(context))(remaining)?;
 
-    // TODO: Consume trailing formulas
     let source = get_consumed(input, remaining);
 
     Ok((
@@ -74,6 +73,12 @@ pub fn org_mode_table<'b, 'g, 'r, 's>(
     ))
 }
 
+#[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
+pub(crate) fn detect_table<'s>(input: OrgSource<'s>) -> Res<OrgSource<'s>, ()> {
+    tuple((start_of_line, space0, tag("|")))(input)?;
+    Ok((input, ()))
+}
+
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
 fn table_end<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
@@ -84,7 +89,7 @@ fn table_end<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn org_mode_table_row<'b, 'g, 'r, 's>(
+fn org_mode_table_row<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, TableRow<'s>> {
@@ -95,7 +100,7 @@ pub fn org_mode_table_row<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn org_mode_table_row_rule<'b, 'g, 'r, 's>(
+fn org_mode_table_row_rule<'b, 'g, 'r, 's>(
     _context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, TableRow<'s>> {
@@ -112,7 +117,7 @@ pub fn org_mode_table_row_rule<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn org_mode_table_row_regular<'b, 'g, 'r, 's>(
+fn org_mode_table_row_regular<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, TableRow<'s>> {
@@ -132,7 +137,7 @@ pub fn org_mode_table_row_regular<'b, 'g, 'r, 's>(
 }
 
 #[cfg_attr(feature = "tracing", tracing::instrument(ret, level = "debug"))]
-pub fn org_mode_table_cell<'b, 'g, 'r, 's>(
+fn org_mode_table_cell<'b, 'g, 'r, 's>(
     context: RefContext<'b, 'g, 'r, 's>,
     input: OrgSource<'s>,
 ) -> Res<OrgSource<'s>, TableCell<'s>> {