192 Commits

Author SHA1 Message Date
Tom Alexander
fcd63b1231 fixup
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 21:58:41 -04:00
Tom Alexander
743b4c9982 fixup
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 21:55:20 -04:00
Tom Alexander
7eccf8fccd Remove dependency on run-docker-image task.
Some checks failed
rust-test Build rust-test has failed
rust-build Build rust-build has succeeded
2023-08-31 21:48:06 -04:00
Tom Alexander
a40a504f94 Switch to using read-only root in docker containers.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 21:23:51 -04:00
Tom Alexander
80d77ff5d6 Fix handling of unicode in compare tests.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 20:20:00 -04:00
Tom Alexander
ee92049e5d Merge branch 'special_block'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 19:19:29 -04:00
Tom Alexander
510985e97c Fix greater blocks by preventing nesting even if they are not the immediate parent. 2023-08-31 19:17:46 -04:00
Tom Alexander
949d0989f4 Add a test showing the current parser is broken with deeply nested greater blocks. 2023-08-31 19:15:28 -04:00
Tom Alexander
2a4d22bdd4 Add test for nesting of greater blocks.
This shows that greater blocks of different types can be directly nested even if they are both special blocks (as long as they are different special blocks).
2023-08-31 19:09:38 -04:00
Tom Alexander
7a903acedc Support special blocks in compare. 2023-08-31 19:04:44 -04:00
Tom Alexander
5171326d63 Create a test for special blocks. 2023-08-31 19:02:18 -04:00
Tom Alexander
67f79aeb51 Remove context from detect plain list.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 18:53:28 -04:00
Tom Alexander
b2383d9f93 Fix footnote definition performance.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
We were re-parsing each footnote definition in the footnote definition exit matcher which causes their contents to get re-parsed. This compounds with long lists of footnote definitions.
2023-08-31 18:47:23 -04:00
Tom Alexander
9e2a323f6f Merge branch 'trailing_whitespace_at_end_of_file'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 17:29:52 -04:00
Tom Alexander
0fcb3f73f9 Move handling of contentless item to handle contentless description item. 2023-08-31 17:25:42 -04:00
Tom Alexander
bfc9e7f58d When re-parsing last item in list, only disable white space consumption for last element in item. 2023-08-31 17:08:21 -04:00
Tom Alexander
b5f0521b56 Only consume trailing whitespace when not exiting for all objects. 2023-08-31 15:45:31 -04:00
Tom Alexander
2048d8f0b6 Support comments at the end of the line in diary sexp.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 15:31:54 -04:00
Tom Alexander
466716881e Fix macros consuming whitespace even when the exit matcher is calling for an exit.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-31 15:11:29 -04:00
Tom Alexander
eb9c582fa5 '>' is allowed as a post character in text markup.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 23:57:15 -04:00
Tom Alexander
214e895d85 Support backslash at the end of a macro. 2023-08-29 23:42:16 -04:00
Tom Alexander
db3086743c Publish version 0.1.4.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 23:28:17 -04:00
Tom Alexander
207a0546b0 Run full test suite despite default feature selection.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 23:23:31 -04:00
Tom Alexander
e9480fd156 Disable the compare tests when the compare feature is disabled.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 23:17:46 -04:00
Tom Alexander
28aca041f7 Publish version 0.1.3.
Some checks failed
rustfmt Build rustfmt has failed
rust-test Build rust-test has failed
rust-build Build rust-build has succeeded
2023-08-29 23:05:36 -04:00
Tom Alexander
d82def2a70 Merge branch 'compare_improvements'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 22:58:42 -04:00
Tom Alexander
d471f7178b Wrap the components of headlines in diff layers. 2023-08-29 22:57:08 -04:00
Tom Alexander
2c5c26c55f Allow diff layers that are not associated with tokens. 2023-08-29 22:47:40 -04:00
Tom Alexander
7944659802 Compare headline level. 2023-08-29 22:11:56 -04:00
Tom Alexander
58aca53144 Move get_property into util. 2023-08-29 22:07:23 -04:00
Tom Alexander
6f2d90162b Do not use the plain text parser for property drawers.
This additional exit condition was causing property drawers to parse incorrectly.
2023-08-29 22:03:20 -04:00
Tom Alexander
f170a557ed Use character offsets in diff. 2023-08-29 21:49:16 -04:00
Tom Alexander
eaa38ce772 Include an error message for failed bounds checking.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 21:27:54 -04:00
Tom Alexander
a6d742a536 Fix handling line breaks after divider in description lists. 2023-08-29 21:27:54 -04:00
Tom Alexander
45be9e7bde Merge branch 'description_list'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 19:36:45 -04:00
Tom Alexander
f6c895319f Add support for diffing description lists. 2023-08-29 19:35:54 -04:00
Tom Alexander
2682779534 Add support for parsing description lists. 2023-08-29 18:13:55 -04:00
Tom Alexander
b48d472546 Fix build.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 18:07:35 -04:00
Tom Alexander
ea6faf728c Add test for description list. 2023-08-29 18:07:35 -04:00
Tom Alexander
f4ea1b7303 Merge branch 'accuracy_fixes_from_feeding_large_documents'
Some checks failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has failed
2023-08-29 17:49:19 -04:00
Tom Alexander
80b55fdd45 Consume trailing whitespace for macro. 2023-08-29 17:42:58 -04:00
Tom Alexander
f426e32798 Do not include trailing punctuation or whitespace in plain links. 2023-08-29 17:35:04 -04:00
Tom Alexander
66037356c5 Add test showing plain links ending in punctuation currently do not parse correctly. 2023-08-29 17:24:44 -04:00
Tom Alexander
1bcd1895c0 Allow affiliating keywords with regular keywords. 2023-08-29 17:19:13 -04:00
Tom Alexander
e3d38cfbe2 Add a test for floating affiliated keywords. 2023-08-29 17:05:19 -04:00
Tom Alexander
2ba0dc49be Support export affiliated keywords. 2023-08-29 17:01:35 -04:00
Tom Alexander
9df40fb13f Only allow specific keywords for affiliated keywords. 2023-08-29 16:56:07 -04:00
Tom Alexander
cc671925db Support empty sections under headings. 2023-08-29 16:07:43 -04:00
Tom Alexander
950baa9d5d Only allow a single section under a heading. 2023-08-29 16:03:13 -04:00
Tom Alexander
56865c68fc Do not allow plain links without a path. 2023-08-29 15:44:04 -04:00
Tom Alexander
f592b73ae7 Merge branch 'reduce_context_usage_in_exit_matchers'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 15:17:16 -04:00
Tom Alexander
3206027b96 Add all entities. 2023-08-29 15:16:22 -04:00
Tom Alexander
3e6df7ba78 Print character offset from rust's parse perspective during compare. 2023-08-29 14:40:58 -04:00
Tom Alexander
ac313d093e Improve error handling in compare. 2023-08-29 14:20:53 -04:00
Tom Alexander
f376f1cf8e Add a test for empty sections. 2023-08-29 14:10:26 -04:00
Tom Alexander
f21385a901 Add a helper function for logging during debugging.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 14:03:14 -04:00
Tom Alexander
1d06d95bb1 Add a minimum stars figure to heading parser to reduce re-parses. 2023-08-29 14:01:28 -04:00
Tom Alexander
bfc88c1d1b Use a detect_headline function instead of the full headline parse for the section_end exit matcher.
This shaved 2 seconds off the first 800 lines of org-mode/doc/org-guide.org.
2023-08-29 11:35:54 -04:00
Tom Alexander
f29720e5b9 Switch to using a type for bracket depth.
This is to make changing the type easier in the future.
2023-08-29 11:18:15 -04:00
Tom Alexander
27a9b5aeb1 Switch to i16 for backet depth count.
This is having a measurable performance increase. 32k bracket depth should be enough for any non-malicious document.
2023-08-29 11:14:50 -04:00
Tom Alexander
8051c3d2b7 Remove line number tracking.
The documentation was incorrect, none of the org-mode elements have a line number restriction for their contents.
2023-08-29 11:09:28 -04:00
Tom Alexander
bd97d2f69d Switch to i32 for tracking bracket depth. 2023-08-29 11:07:00 -04:00
Tom Alexander
14b1d0526c Manually implement Debug and make convert_error more generic.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-29 10:58:05 -04:00
Tom Alexander
288350daef Iterate over the bytes instead of characters when counting brackets.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-28 03:52:21 -04:00
Tom Alexander
c683516620 Switch inline source blocks to using bracket depth from OrgSource instead of from the context.
This is for the same reasons as footnote references.
2023-08-28 03:04:33 -04:00
Tom Alexander
e731e8ff6b Switch inline babel call to using bracket depth from OrgSource instead of from the context.
This is for the same reasons as footnote references.
2023-08-28 03:04:33 -04:00
Tom Alexander
4c2037ec44 Switch subscript and superscript to using bracket depth from OrgSource instead of from the context.
This is for the same reasons as footnote references.
2023-08-28 03:04:33 -04:00
Tom Alexander
a46b358549 Switch citations to using bracket depth from OrgSource instead of from the context.
This is for the same reasons as footnote references.
2023-08-28 03:04:32 -04:00
Tom Alexander
ec813e3b3f Switch to using bracket depth from OrgSource instead of from the context for footnote references.
It is currently unknown if this will produce a performance increase, but unless it has a significant performance penalty we are going to go forward with this change because it makes it more explicit which values need to be read deeply from other elements (therefore needing to be in the context) vs values that can be bound to the exit matcher since they are only used within the confines of the current element.

I suspect we will get a performance boost since it will be reducing the nodes that need to be walked in the context but maintaining bracket depth count over the entire document instead of only inside elements that need balanced brackets could cost us.
2023-08-28 03:04:32 -04:00
Tom Alexander
f11f7bcc73 Keep track of bracket, brace, and parenthesis depth when iterating over the OrgSource. 2023-08-28 01:18:46 -04:00
Tom Alexander
9e0e5f6f0a Remove line number limit for LaTeX fragments. 2023-08-28 01:18:46 -04:00
Tom Alexander
16e788c36c Add tests for LaTeX fragments and text markup that span more than three lines.
The documentation currently states that the body for these cannot span more than three lines but that is not the behavior I am seeing from emacs in practice. Waiting on a mailing list response to tell me if this is a documentation error or a parser error.
2023-08-28 01:18:46 -04:00
Tom Alexander
b35d785e73 Fix tracing in the run_docker_compare.bash script.
Some checks failed
rust-build Build rust-build has failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
2023-08-28 01:18:45 -04:00
Tom Alexander
1952d175c0 Record line number in OrgSource.
This will be used for elements who have limits on the number of lines inside of them. This includes LaTeX fragment bodies and text markup.
2023-08-28 01:18:45 -04:00
Tom Alexander
20c17c40be Switch greater blocks to using name provided when building exit matcher instead of from context. 2023-08-28 01:18:45 -04:00
Tom Alexander
b6b869df25 Minor improvement to error message in diff.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-28 01:05:09 -04:00
Tom Alexander
18a396b7cb Remove deprecated tests.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 23:25:50 -04:00
Tom Alexander
085490476e Fix make dockertest.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 22:41:55 -04:00
Tom Alexander
9c9964c66f Add lt and gt entities.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 22:15:23 -04:00
Tom Alexander
1a3e26c148 Update plain list greater block exit matcher priority test to match blog post.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
It is still testing the same thing, but I changed the contents a bit to match what is being used in my blog post.
2023-08-27 21:03:16 -04:00
Tom Alexander
e9e6a8ff64 Merge branch 'clean_up_docker'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 18:03:51 -04:00
Tom Alexander
b124317f30 Fix up scripts to handle the changes to the docker containers. 2023-08-27 18:03:37 -04:00
Tom Alexander
ad389f0776 Remove volumes in the clean step.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 17:53:51 -04:00
Tom Alexander
75dfc7f812 Depend on build when using the docker images. 2023-08-27 17:51:57 -04:00
Tom Alexander
c17de8ef5e Set up the same mounts when running make shell. 2023-08-27 17:50:33 -04:00
Tom Alexander
378b6bb391 Update the run targets for the Makefiles for the docker containers.
This was previously using the standard docker makefile I use as a starting point for all of my docker makefiles. Now it will properly mount the source directory.
2023-08-27 17:46:36 -04:00
Tom Alexander
cc86591a6c Support the debug/dev profile in the perf script.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 17:02:12 -04:00
Tom Alexander
f25dbc1d7c Add a script for testing organic parse times.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
This is not meant to produce publishable or comparable benchmarks. Such a script would have to run many iterations with the input already loaded into memory, proper prioritization via nice/ionice, and have a warm-up phase. This is just automating a basic test I am frequently running to compare parse times when investigating performance issues.
2023-08-27 16:56:32 -04:00
Tom Alexander
daee50c160 Merge branch 'dynamic_block_test'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-27 16:23:28 -04:00
Tom Alexander
3e143796f7 Compare heading todo keywords.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
This only handles the default case where the only valid TODO keywords are TODO and DONE.
2023-08-27 15:56:08 -04:00
Tom Alexander
9cc5e63c1b Compare heading tags.
Some checks failed
rust-test Build rust-test has failed
rust-build Build rust-build has succeeded
2023-08-25 07:05:59 -04:00
Tom Alexander
be6197e4c7 Store the tags in the heading. 2023-08-25 06:20:06 -04:00
Tom Alexander
2d4e54845b Add support for parsing tags in headlines. 2023-08-25 06:13:29 -04:00
Tom Alexander
d5ea650b96 Add a test for dynamic blocks. 2023-08-25 05:36:57 -04:00
Tom Alexander
60363579b5 Merge branch 'plain_list_content_on_next_line'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 05:28:14 -04:00
Tom Alexander
1b678fe81f Add tests for detect_plain_list. 2023-08-25 05:27:09 -04:00
Tom Alexander
bfea828e62 Update detect_plain_list to support line breaks. 2023-08-25 05:27:08 -04:00
Tom Alexander
bc5745a95f Add support for list items with a line break before their contents. 2023-08-25 05:18:26 -04:00
Tom Alexander
efa372a9e9 Add a test case that breaks the current parser. 2023-08-25 04:39:58 -04:00
Tom Alexander
2fb57daaec Move the table cell object parser into the object parser file.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 04:35:54 -04:00
Tom Alexander
3a38f4cd35 Add support for the ast entity.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 04:28:24 -04:00
Tom Alexander
45e16fea2d Honor the NO_COLOR environment variable.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 04:25:39 -04:00
Tom Alexander
5134cece7b Add color to compare output.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 04:09:52 -04:00
Tom Alexander
19432d91ab Get the emacs and org-mode versions when launching the compare script.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 03:11:19 -04:00
Tom Alexander
16a107eebb Update org-mode version. 2023-08-25 02:56:28 -04:00
Tom Alexander
77348b560c Parameterize the emacs and org-mode versions in the dockerfiles.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-build Build rust-build has succeeded
rust-test Build rust-test has succeeded
2023-08-25 02:03:35 -04:00
Tom Alexander
fc79507ef3 Merge branch 'plain_list_perf_investigation'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 01:10:04 -04:00
Tom Alexander
9c1e6ccc97 Add a detect_element function.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
This is an optimization. When you have something like plain text which ends when it hits the next element, we only need to parse enough to detect that an element is about to occur. For elements like plain lists, this is as simple as parsing a line starting with optional whitespace and then a bullet, which avoids parsing the entire plain list tree. The benefit is most noticeable in deeply nested plain lists.
2023-08-25 01:07:53 -04:00
Tom Alexander
0dbc8f0925 Remove redundant exit matcher checks.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-25 00:01:56 -04:00
Tom Alexander
02fe10fba3 Move objects to a lower exit class.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
Paragraph's exit matcher which detects elements was causing the plain list parser to exit after the first item was parsed which was causing significant amounts of re-parsing.
2023-08-24 23:34:23 -04:00
Tom Alexander
33d7ae03d1 Add a TODO.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-24 21:35:34 -04:00
Tom Alexander
03faa7257f Move the indent level for plain list's exit matcher to const fn instead of grabbing from the context.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
This made a slight improvement to performance.
2023-08-24 20:50:24 -04:00
Tom Alexander
ae3510abd5 Do not cast lesser block name to lowercase at runtime.
This reduced the runtime of my problematic test case from 6.9 seconds to 6 seconds.
2023-08-24 20:10:43 -04:00
fluxcdbot
ad3f47864a CI: autofix rust code.
Some checks failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has failed
2023-08-24 23:43:41 +00:00
Tom Alexander
533ef2a9a8 Merge branch 'wrapped_input'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-24 19:40:55 -04:00
Tom Alexander
cf37bc4111 Remove unnecessary context from some util functions.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-24 19:29:00 -04:00
Tom Alexander
e5224cda63 Removing dead code.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-24 18:40:25 -04:00
Tom Alexander
64e3481660 Update get_consumed to use the new wrapped input type. 2023-08-24 18:33:40 -04:00
Tom Alexander
32071ce74d Fix handling of start of line in OrgSource.
All checks were successful
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-24 18:08:16 -04:00
Tom Alexander
e84e2b5147 Update tests to compile again. 2023-08-24 17:15:24 -04:00
Tom Alexander
3348807a05 Eliminate the document root context element. 2023-08-24 17:01:12 -04:00
Tom Alexander
720afa5d32 Update getting the previous character and previous line.
This can be done a lot more efficiently now that we are keeping track of this information in the wrapped input type instead of having to fetch to the original document out of the context tree.
2023-08-24 16:56:07 -04:00
Tom Alexander
dab598e5e7 Convert all functions to using the wrapped input type.
Some checks failed
rust-test Build rust-test has failed
rust-build Build rust-build has failed
2023-08-24 16:06:29 -04:00
Tom Alexander
b7a5dd48ea Impl missing traits. 2023-08-22 23:32:27 -04:00
Tom Alexander
c475dce6da Fix lifetime issue. 2023-08-22 23:14:23 -04:00
Tom Alexander
6d1675fa00 Lifetime issue. 2023-08-22 22:57:44 -04:00
Tom Alexander
cda49c628c Move the wrapped input into the parser. 2023-08-22 22:33:50 -04:00
Tom Alexander
65b87bd65d Merge remote-tracking branch 'input/main' into wrapped_input 2023-08-22 22:26:55 -04:00
Tom Alexander
5a7f34b63e Prepare for merging into Organic. 2023-08-22 22:24:35 -04:00
Tom Alexander
edff1e089d Implement text since line break. 2023-08-22 22:18:44 -04:00
Tom Alexander
bc29f1dfc0 Add slicing tests. 2023-08-22 21:38:50 -04:00
Tom Alexander
e4656cddf6 Implement slice, take, and compare. 2023-08-22 21:25:13 -04:00
Tom Alexander
1e3dadd458 Wrap the input. 2023-08-22 17:24:26 -04:00
Tom Alexander
2ec055af5a Very simple setup. 2023-08-22 17:22:13 -04:00
Tom Alexander
6823db5c60 Initial commit. 2023-08-22 17:11:45 -04:00
Tom Alexander
21e1ceb8e0 Merge branch 'add_performance_check_scripts'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-22 14:21:44 -04:00
Tom Alexander
655af88cdf Add scripts for running perf and callgrind. 2023-08-22 14:21:27 -04:00
Tom Alexander
8561fdc1bd Make the autogen prefix fully integrated into the test name.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-21 00:14:10 -04:00
Tom Alexander
f2089257b0 Re-enable disabled test.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-build Build rust-build has succeeded
rust-test Build rust-test has succeeded
The latest code in org-mode has been fixed.
2023-08-21 00:08:26 -04:00
Tom Alexander
09821c8898 Prefix the automatically generated tests.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-20 23:53:11 -04:00
Tom Alexander
69ecfd2646 Move all the specific-token tests into subfolders.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-20 23:38:47 -04:00
Tom Alexander
8162f03051 Put all trailing whitespace ownership test cases into the automated tests.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
Notes for this investigation moved to cba1d1e988/notes/plain_list_ownership_notes.org .

Mailing list thread on the investigation: https://list.orgmode.org/9372527e-3852-419e-936a-7b4dd38cc847@app.fastmail.com/ .
2023-08-20 16:03:31 -04:00
Tom Alexander
d8c3285e3c Add --init flag to docker run.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-build Build rust-build has succeeded
rust-test Build rust-test has succeeded
I noticed in a separate project that ctrl+c was not being honored under --init was passed, so I'm adding it in here.
2023-08-19 02:51:00 -04:00
Tom Alexander
5db6cd617e Improve test cases for plain list ownership. 2023-08-19 02:30:31 -04:00
Tom Alexander
4cd3697fb0 Update org-mode version in dockerfile. 2023-08-18 23:20:29 -04:00
Tom Alexander
2cd6f736c2 Fix building without compare feature.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-17 00:13:25 -04:00
fluxcdbot
5686256039 CI: autofix rust code.
Some checks failed
rust-build Build rust-build has failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
2023-08-17 04:05:48 +00:00
Tom Alexander
7cf1b2d2b8 Disable the failing plain list whitespace ownership test.
Some checks failed
rust-build Build rust-build has failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
2023-08-17 00:03:05 -04:00
Tom Alexander
b848d7be73 Merge branch 'no_files' 2023-08-16 23:57:58 -04:00
Tom Alexander
74f4aa8d33 Remove dependency on files for running compare.
The tests still use files since they get the test name from a file but compare does the same action via stdin so it can operator on any org source.
2023-08-16 23:56:05 -04:00
Tom Alexander
4776898894 Merge branch 'fix_plain_list'
Some checks failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has failed
rust-build Build rust-build has succeeded
2023-08-16 21:06:54 -04:00
Tom Alexander
8e95ce6368 Add notes about plain list trailing blank line ownership investigation. 2023-08-16 20:24:00 -04:00
Tom Alexander
6c9c304f37 Re-enable disabled test. 2023-08-16 17:39:10 -04:00
Tom Alexander
7fafbfb6bb Do not consume whitespace in the final plain list item. 2023-08-16 17:37:19 -04:00
Tom Alexander
56281633f3 Support blank link in plain_list_item_end, move exit matcher to end of loop in plain_list, and maybe consume trailing whitespace in plain_list_item. 2023-08-16 17:09:06 -04:00
Tom Alexander
823c33ef8e Reduce use of expect in main.rs 2023-08-16 16:37:14 -04:00
Tom Alexander
e5e5120a10 Move telemetry handling to the tracing-specific main function.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
This is so main_body can exit with an error at any time without missing the shutdown_telemetry function. This does not catch panics.
2023-08-16 16:05:24 -04:00
Tom Alexander
7df393f31d Make a new naive implementation of plain_list_item.
Still need to update plain_list_item_end and handle the whitespace ownership issues, but starting from a simplified state will help.
2023-08-16 16:05:24 -04:00
Tom Alexander
72d5f8f35c Make a new naive implementation of plain_list. 2023-08-16 16:05:24 -04:00
Tom Alexander
dae46adc12 Feature-gate tracing import.
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-14 23:33:38 -04:00
Tom Alexander
d0dc737c79 Merge branch 'plain_list_whitespace_ownership_issue'
Some checks failed
rust-build Build rust-build has failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
2023-08-14 23:20:37 -04:00
Tom Alexander
1c9877015d Disable the test showing my plain list implementation is broken. 2023-08-14 23:20:28 -04:00
Tom Alexander
2938d5809a Use the rust cache for make dockertest. 2023-08-14 23:17:38 -04:00
Tom Alexander
f7ec89858d Add notes about optimization ideas. 2023-08-14 23:16:23 -04:00
Tom Alexander
67b4dfdce6 Merge branch 'tracing_fixes'
Some checks failed
rust-build Build rust-build has failed
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
2023-08-14 22:12:05 -04:00
Tom Alexander
63d092c83d Group the two traces per compare into one trace. 2023-08-14 22:10:58 -04:00
Tom Alexander
a7b298eeec Fix lesser block exit priority.
The paragraph end was matching text inside lesser blocks.
2023-08-14 17:32:10 -04:00
Tom Alexander
1bbfbc3164 Add additional tracing to lesser block. 2023-08-14 17:32:09 -04:00
Tom Alexander
2bcc3f0599 Fix reporting of jaeger traces when diff does not match.
The early exit was causing some traces to not be reported.
2023-08-14 17:32:09 -04:00
Tom Alexander
b93a12c32c Add support for escaped double quotes in sexp. 2023-08-14 16:55:04 -04:00
Tom Alexander
df3045e424 Merge branch 'script_improvement'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-test Build rust-test has succeeded
rust-build Build rust-build has succeeded
2023-08-14 16:13:30 -04:00
Tom Alexander
72b8fec1be Add support for tracing in run_docker_compare.bash. 2023-08-14 16:12:31 -04:00
Tom Alexander
ab17904b1c Clean up run_integration_test.bash. 2023-08-14 15:53:17 -04:00
Tom Alexander
306878c95d Clean up run_docker_integration_test.bash 2023-08-14 15:50:05 -04:00
Tom Alexander
5768c8acda Add a script to run compare using the docker image. 2023-08-14 15:30:13 -04:00
Tom Alexander
e28290ed79 Merge branch 'source_based_tests'
All checks were successful
rustfmt Build rustfmt has succeeded
rust-build Build rust-build has succeeded
rust-test Build rust-test has succeeded
2023-08-14 14:14:26 -04:00
Tom Alexander
fbabf60559 Add ignore to test export_snippet_paragraph_break_precedence. 2023-08-14 14:01:00 -04:00
Tom Alexander
92abac37e2 s/precedent/precedence/
I used the wrong word. This is referring to the priority between paragraphs ending vs export snippets ending, not a reference to something occurring in the past.
2023-08-14 13:57:01 -04:00
Tom Alexander
899073e54f Update to the latest org-mode. 2023-08-14 13:33:05 -04:00
Tom Alexander
eb379af78d Switch export snippet to use exit matchers. 2023-08-14 13:13:32 -04:00
Tom Alexander
422804d846 Add script for running specific tests inside docker.
Some checks failed
rust-build Build rust-build has failed
rust-test Build rust-test has failed
2023-08-14 12:21:15 -04:00
Tom Alexander
cc83431d62 Consume trailing whitespace for property drawers.
Some checks failed
rust-build Build rust-build has failed
rust-test Build rust-test has failed
This is a change between the org-mode in emacs 29.1 and the org-mode currently in main.
2023-08-14 11:57:12 -04:00
Tom Alexander
00354ccc20 Add a volume for cargo cache.
This is to be a good citizen by not downloading all the rust dependencies every time I run the tests locally. Unfortunately, it will still compile all the dependencies each time, but that is a local operation.
2023-08-14 10:57:48 -04:00
Tom Alexander
b75eed6b1e Enable tests that were disabled before.
Some checks are pending
rust-test Build rust-test has started
rust-build Build rust-build has succeeded
2023-08-13 02:21:02 -04:00
Tom Alexander
e33ec4a02c Add support for reading begin/end bounds in the new standard-properties format. 2023-08-13 02:21:02 -04:00
Tom Alexander
f7afcec824 Add support for hash notation in the elisp parser. 2023-08-13 02:21:02 -04:00
Tom Alexander
cf0991fdff Add support for parsing vectors in the elisp parser. 2023-08-13 02:21:02 -04:00
Tom Alexander
d1e0ee831c Switch to installing emacs and org-mode from source in test container.
This is to integrate fixes that have been committed to org-mode but have not made it into emacs, while also getting the latest emacs on alpine.
2023-08-13 02:21:01 -04:00
Tom Alexander
34985c9045 Add makefile target for running the tests inside the docker container.
Some checks failed
rust-build Build rust-build has failed
rust-test Build rust-test has failed
rustfmt Build rustfmt has succeeded
2023-08-13 02:20:16 -04:00
Tom Alexander
7da09fea74 Switch to specifying timeouts instead of timeout in tekton pipelinerun. 2023-08-13 02:20:16 -04:00
Tom Alexander
fc28e3b514 Add a test for trailing blank lines after paragraphs.
Some checks failed
rust-test Build rust-test has failed
rustfmt Build rustfmt has succeeded
rust-build Build rust-build has succeeded
The behavior in emacs does not match the description in the org-mode documentation. I have sent an email to the org-mode mailing list and I am waiting their response so I can adjust (or not adjust) my parser accordingly.
2023-08-11 01:37:04 -04:00
Tom Alexander
df5ee5af16 Explicitly list which files to include in the cargo package.
Some checks failed
rust-build Build rust-build has failed
rust-test Build rust-test has failed
rustfmt Build rustfmt has succeeded
We are including a bunch of files that are not needed for running the rust code. This excludes them to be a better citizen to both crates.io and all users of this package.
2023-08-11 00:11:54 -04:00
175 changed files with 5303 additions and 2311 deletions

View File

@@ -1,3 +1,4 @@
**/.git
target
Cargo.lock
notes/

View File

@@ -14,10 +14,6 @@ spec:
- name: path-to-dockerfile
description: The path to the Dockerfile
type: string
- name: command
type: array
description: Command to run.
default: []
tasks:
- name: report-pending
taskRef:
@@ -92,8 +88,6 @@ spec:
runAfter:
- build-image
params:
- name: command
value: ["$(params.command[*])"]
- name: args
value: ["--no-default-features"]
- name: docker-image
@@ -109,8 +103,6 @@ spec:
runAfter:
- run-image-none
params:
- name: command
value: ["$(params.command[*])"]
- name: args
value: ["--no-default-features", "--features", "tracing"]
- name: docker-image
@@ -126,8 +118,6 @@ spec:
runAfter:
- run-image-tracing
params:
- name: command
value: ["$(params.command[*])"]
- name: args
value: ["--no-default-features", "--features", "compare"]
- name: docker-image
@@ -143,8 +133,6 @@ spec:
runAfter:
- run-image-compare
params:
- name: command
value: ["$(params.command[*])"]
- name: args
value: []
- name: docker-image
@@ -160,8 +148,6 @@ spec:
runAfter:
- run-image-default
params:
- name: command
value: ["$(params.command[*])"]
- name: args
value: ["--no-default-features", "--features", "tracing,compare"]
- name: docker-image
@@ -256,5 +242,3 @@ spec:
value: docker/organic_build/
- name: path-to-dockerfile
value: docker/organic_build/Dockerfile
- name: command
value: [cargo, build]

View File

@@ -4,6 +4,10 @@ metadata:
name: rust-test
spec:
pipelineSpec:
timeouts:
pipeline: "2h0m0s"
tasks: "1h0m40s"
finally: "0h30m0s"
params:
- name: image-name
description: The name for the built image
@@ -14,14 +18,6 @@ spec:
- name: path-to-dockerfile
description: The path to the Dockerfile
type: string
- name: command
type: array
description: Command to run.
default: []
- name: args
type: array
description: Arguments passed to command.
default: []
tasks:
- name: do-stuff
taskSpec:
@@ -103,22 +99,41 @@ spec:
runAfter:
- fetch-repository
- name: run-image
taskRef:
name: run-docker-image
taskSpec:
metadata: {}
stepTemplate:
name: ""
workingDir: "$(workspaces.source.path)"
workspaces:
- name: source
mountPath: /source
- name: cargo-cache
mountPath: /usr/local/cargo/registry
optional: true
steps:
- name: run
image: "$(params.IMAGE)"
command: []
args:
[
--no-default-features,
--features,
compare,
--no-fail-fast,
--lib,
--test,
test_loader,
]
workspaces:
- name: source
workspace: git-source
- name: cargo-cache
workspace: cargo-cache
params:
- name: IMAGE
value: "$(params.image-name):$(tasks.fetch-repository.results.commit)"
runAfter:
- build-image
params:
- name: command
value: ["$(params.command[*])"]
- name: args
value: ["$(params.args[*])"]
- name: docker-image
value: "$(params.image-name):$(tasks.fetch-repository.results.commit)"
finally:
- name: report-success
when:
@@ -201,7 +216,6 @@ spec:
secret:
secretName: harbor-plain
serviceAccountName: build-bot
timeout: 240h0m0s
params:
- name: image-name
value: "harbor.fizz.buzz/private/organic-test"
@@ -209,7 +223,3 @@ spec:
value: docker/organic_test/
- name: path-to-dockerfile
value: docker/organic_test/Dockerfile
- name: command
value: [cargo, test]
- name: args
value: [--lib, --test, test_loader]

View File

@@ -14,14 +14,6 @@ spec:
- name: path-to-dockerfile
description: The path to the Dockerfile
type: string
- name: rustfmt-command
type: array
description: Command to run rustfmt.
default: []
- name: rustfmt-args
type: array
description: Arguments passed to rustfmt.
default: []
- name: GIT_USER_NAME
description: The username for git
type: string
@@ -119,10 +111,6 @@ spec:
runAfter:
- build-image
params:
- name: command
value: ["$(params.rustfmt-command[*])"]
- name: args
value: ["$(params.rustfmt-args[*])"]
- name: docker-image
value: "$(params.image-name):$(tasks.fetch-repository.results.commit)"
- name: cargo-fix
@@ -240,7 +228,3 @@ spec:
value: docker/cargo_fmt/
- name: path-to-dockerfile
value: docker/cargo_fmt/Dockerfile
- name: command
value: [cargo, fmt]
- name: args
value: []

View File

@@ -1,6 +1,6 @@
[package]
name = "organic"
version = "0.1.2"
version = "0.1.4"
authors = ["Tom Alexander <tom@fizz.buzz>"]
description = "An org-mode parser."
edition = "2021"
@@ -10,6 +10,11 @@ readme = "README.md"
keywords = ["emacs", "org-mode"]
categories = ["parsing"]
resolver = "2"
include = [
"LICENSE",
"**/*.rs",
"Cargo.toml"
]
[lib]
name = "organic"
@@ -34,10 +39,18 @@ tracing-subscriber = { version = "0.3.17", optional = true, features = ["env-fil
walkdir = "2.3.3"
[features]
default = ["compare", "tracing"]
default = []
compare = []
tracing = ["dep:opentelemetry", "dep:opentelemetry-otlp", "dep:opentelemetry-semantic-conventions", "dep:tokio", "dep:tracing", "dep:tracing-opentelemetry", "dep:tracing-subscriber"]
[profile.release]
# Optimized build for any sort of release.
[profile.release-lto]
inherits = "release"
lto = true
strip = "symbols"
# Profile for performance testing with the "perf" tool. Notably keeps debug enabled and does not strip symbols to make reading the perf output easier.
[profile.perf]
inherits = "release"
lto = true
debug = true

View File

@@ -35,22 +35,32 @@ clean:
.PHONY: test
test:
> cargo test --lib --test test_loader -- --test-threads $(TESTJOBS)
> cargo test --no-default-features --features compare --no-fail-fast --lib --test test_loader -- --test-threads $(TESTJOBS)
.PHONY: dockertest
dockertest:
> $(MAKE) -C docker/organic_test
> docker run --init --rm -i -t --read-only -v "$$(readlink -f ./):/source:ro" --mount type=tmpfs,destination=/tmp --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target -w /source organic-test --no-default-features --features compare --no-fail-fast --lib --test test_loader -- --test-threads $(TESTJOBS)
.PHONY: dockerclean
dockerclean:
# Delete volumes created for running the tests in docker. This does not touch anything related to the jaeger docker container.
> docker volume rm cargo-cache rust-cache
.PHONY: integrationtest
integrationtest:
> cargo test --no-fail-fast --test test_loader -- --test-threads $(TESTJOBS)
> cargo test --no-default-features --features compare --no-fail-fast --test test_loader -- --test-threads $(TESTJOBS)
.PHONY: unittest
unittest:
> cargo test --lib -- --test-threads $(TESTJOBS)
> cargo test --no-default-features --lib -- --test-threads $(TESTJOBS)
.PHONY: jaeger
jaeger:
# 4317 for OTLP gRPC, 4318 for OTLP HTTP. We currently use gRPC but I forward both ports regardless.
#
# These flags didn't help even though they seem like they would: --collector.otlp.grpc.max-message-size=10000000 --collector.queue-size=20000 --collector.num-workers=100
> docker run -d --rm --name organicdocker -p 4317:4317 -p 4318:4318 -p 16686:16686 -e COLLECTOR_OTLP_ENABLED=true jaegertracing/all-in-one:1.47 --collector.grpc-server.max-message-size=10000000
# These flags didn't help even though they seem like they would: --collector.queue-size=20000 --collector.num-workers=100
> docker run -d --rm --name organicdocker --read-only -p 4317:4317 -p 4318:4318 -p 16686:16686 -e COLLECTOR_OTLP_ENABLED=true jaegertracing/all-in-one:1.47 --collector.grpc-server.max-message-size=20000000 --collector.otlp.grpc.max-message-size=20000000
.PHONY: jaegerweb
jaegerweb:

View File

@@ -1,10 +1,16 @@
#[cfg(feature = "compare")]
use std::env;
#[cfg(feature = "compare")]
use std::fs::File;
#[cfg(feature = "compare")]
use std::io::Write;
#[cfg(feature = "compare")]
use std::path::Path;
#[cfg(feature = "compare")]
use walkdir::WalkDir;
#[cfg(feature = "compare")]
fn main() {
let out_dir = env::var("OUT_DIR").unwrap();
let destination = Path::new(&out_dir).join("tests.rs");
@@ -31,6 +37,10 @@ fn main() {
}
}
#[cfg(not(feature = "compare"))]
fn main() {}
#[cfg(feature = "compare")]
fn write_test(test_file: &mut File, test: &walkdir::DirEntry) {
let test_name = test
.path()
@@ -41,6 +51,7 @@ fn write_test(test_file: &mut File, test: &walkdir::DirEntry) {
.strip_suffix(".org")
.expect("Should have .org extension")
.replace("/", "_");
let test_name = format!("autogen_{}", test_name);
if let Some(_reason) = is_expect_fail(test_name.as_str()) {
write!(test_file, "#[ignore]\n").unwrap();
@@ -54,6 +65,7 @@ fn write_test(test_file: &mut File, test: &walkdir::DirEntry) {
.unwrap();
}
#[cfg(feature = "compare")]
fn write_header(test_file: &mut File) {
write!(
test_file,
@@ -69,12 +81,12 @@ use organic::parser::sexp::sexp_with_padding;
.unwrap();
}
#[cfg(feature = "compare")]
fn is_expect_fail(name: &str) -> Option<&str> {
match name {
"drawer_drawer_with_headline_inside" => Some("Apparently lines with :end: become their own paragraph. This odd behavior needs to be investigated more."),
"element_container_priority_footnote_definition_dynamic_block" => Some("Apparently broken begin lines become their own paragraph."),
"paragraphs_paragraph_with_backslash_line_breaks" => Some("The text we're getting out of the parse tree is already processed to remove line breaks, so our comparison needs to take that into account."),
"export_snippet_paragraph_break_precedent" => Some("Emacs 28 has broken behavior so the tests in the CI fail."),
"autogen_greater_element_drawer_drawer_with_headline_inside" => Some("Apparently lines with :end: become their own paragraph. This odd behavior needs to be investigated more."),
"autogen_element_container_priority_footnote_definition_dynamic_block" => Some("Apparently broken begin lines become their own paragraph."),
"autogen_lesser_element_paragraphs_paragraph_with_backslash_line_breaks" => Some("The text we're getting out of the parse tree is already processed to remove line breaks, so our comparison needs to take that into account."),
_ => None,
}
}

View File

@@ -6,7 +6,7 @@ all: build push
.PHONY: build
build:
docker build -t $(IMAGE_NAME) -f Dockerfile ../
docker build -t $(IMAGE_NAME) -f Dockerfile ../../
.PHONY: push
push:
@@ -26,10 +26,11 @@ else
@echo "REMOTE_REPO not defined, not removing from remote repo."
endif
# NOTE: This target will write to folders underneath the git-root
.PHONY: run
run:
docker run --rm -i -t $(IMAGE_NAME)
run: build
docker run --rm --init --read-only --mount type=tmpfs,destination=/tmp -v "$$(readlink -f ../../):/source" --workdir=/source $(IMAGE_NAME)
.PHONY: shell
shell:
docker run --rm -i -t --entrypoint /bin/bash $(IMAGE_NAME)
shell: build
docker run --rm -i -t --entrypoint /bin/sh --mount type=tmpfs,destination=/tmp -v "$$(readlink -f ../../):/source" --workdir=/source $(IMAGE_NAME)

View File

@@ -2,3 +2,5 @@ FROM rustlang/rust:nightly-alpine3.17
RUN apk add --no-cache musl-dev
RUN cargo install --locked --no-default-features --features ci-autoclean cargo-cache
ENTRYPOINT ["cargo", "build"]

View File

@@ -6,7 +6,7 @@ all: build push
.PHONY: build
build:
docker build -t $(IMAGE_NAME) -f Dockerfile ../
docker build -t $(IMAGE_NAME) -f Dockerfile ../../
.PHONY: push
push:
@@ -25,11 +25,13 @@ ifdef REMOTE_REPO
else
@echo "REMOTE_REPO not defined, not removing from remote repo."
endif
docker volume rm cargo-cache
# NOTE: This target will write to folders underneath the git-root
.PHONY: run
run:
docker run --rm -i -t $(IMAGE_NAME)
run: build
docker run --rm --init --read-only --mount type=tmpfs,destination=/tmp -v "$$(readlink -f ../../):/source" --workdir=/source --mount source=cargo-cache,target=/usr/local/cargo/registry $(IMAGE_NAME)
.PHONY: shell
shell:
docker run --rm -i -t --entrypoint /bin/bash $(IMAGE_NAME)
shell: build
docker run --rm -i -t --entrypoint /bin/sh --mount type=tmpfs,destination=/tmp -v "$$(readlink -f ../../):/source" --workdir=/source --mount source=cargo-cache,target=/usr/local/cargo/registry $(IMAGE_NAME)

View File

@@ -1,4 +1,35 @@
FROM rustlang/rust:nightly-alpine3.17
FROM alpine:3.17 AS build
RUN apk add --no-cache build-base musl-dev git autoconf make texinfo gnutls-dev ncurses-dev gawk
RUN apk add --no-cache musl-dev emacs
FROM build AS build-emacs
ARG EMACS_VERSION=emacs-29.1
RUN git clone --depth 1 --branch $EMACS_VERSION https://git.savannah.gnu.org/git/emacs.git /root/emacs
WORKDIR /root/emacs
RUN mkdir /root/dist
RUN ./autogen.sh
RUN ./configure --prefix /usr --without-x --without-sound
RUN make
RUN make DESTDIR="/root/dist" install
FROM build AS build-org-mode
ARG ORG_VERSION=7bdec435ff5d86220d13c431e799c5ed44a57da1
COPY --from=build-emacs /root/dist/ /
RUN mkdir /root/dist
# Savannah does not allow fetching specific revisions, so we're going to have to put unnecessary load on their server by cloning main and then checking out the revision we want.
RUN git clone https://git.savannah.gnu.org/git/emacs/org-mode.git /root/org-mode && git -C /root/org-mode checkout $ORG_VERSION
# RUN mkdir /root/org-mode && git -C /root/org-mode init --initial-branch=main && git -C /root/org-mode remote add origin https://git.savannah.gnu.org/git/emacs/org-mode.git && git -C /root/org-mode fetch origin $ORG_VERSION && git -C /root/org-mode checkout FETCH_HEAD
WORKDIR /root/org-mode
RUN make compile
RUN make DESTDIR="/root/dist" install
FROM rustlang/rust:nightly-alpine3.17
ENV LANG=en_US.UTF-8
RUN apk add --no-cache musl-dev ncurses gnutls
RUN cargo install --locked --no-default-features --features ci-autoclean cargo-cache
COPY --from=build-emacs /root/dist/ /
COPY --from=build-org-mode /root/dist/ /
ENTRYPOINT ["cargo", "test"]

View File

@@ -6,7 +6,7 @@ all: build push
.PHONY: build
build:
docker build -t $(IMAGE_NAME) -f Dockerfile ../
docker build -t $(IMAGE_NAME) -f Dockerfile ../../
.PHONY: push
push:
@@ -25,11 +25,12 @@ ifdef REMOTE_REPO
else
@echo "REMOTE_REPO not defined, not removing from remote repo."
endif
docker volume rm rust-cache cargo-cache
.PHONY: run
run:
docker run --rm -i -t $(IMAGE_NAME)
run: build
docker run --rm --init --read-only --mount type=tmpfs,destination=/tmp -v "$$(readlink -f ../../):/source:ro" --workdir=/source --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target $(IMAGE_NAME) --no-default-features --features compare --no-fail-fast --lib --test test_loader
.PHONY: shell
shell:
docker run --rm -i -t --entrypoint /bin/bash $(IMAGE_NAME)
shell: build
docker run --rm -i -t --entrypoint /bin/sh --mount type=tmpfs,destination=/tmp -v "$$(readlink -f ../../):/source:ro" --workdir=/source --mount source=cargo-cache,target=/usr/local/cargo/registry --mount source=rust-cache,target=/target --env CARGO_TARGET_DIR=/target $(IMAGE_NAME)

1
elisp_snippets/README.md Normal file
View File

@@ -0,0 +1 @@
This folder is for snippets of elisp that are useful for development.

View File

@@ -0,0 +1,3 @@
(dolist (var org-element-affiliated-keywords)
(message "\"%s\"," (downcase var))
)

View File

@@ -0,0 +1,5 @@
(dolist (var org-entities)
(when (listp var)
(message "\"%s\"," (nth 0 var))
)
)

View File

@@ -0,0 +1,27 @@
* Analysis
** Parse start per character
It might help analysis to record how often we start a specific type of parse for each character. For example, at the start of a plain list, if we had a count of how often each character was the start of a parse of a list we could use that to see how often that list is getting re-parsed.
* Optimizations
** Edit whitespace for list items
Whether or not a list item owns the trailing whitespace depends on if it is the last list item in that list. Since we do not know ahead of time if an item is the last item in the list, we have to either re-parse the list item or modify it after parsing.
*** For
We already are modifying the source of some elements after-the-fact with src_rust{set_source()} so this would be more of the same.
*** Against
I'd like to phase out such modifications because they seem hacky and fragile.
** Make detect element function
Some exit matchers are based on when the next element is found. Some elements do not need to be fully parsed to identify that they are a valid element. For example, src_org{1. foo} can already be identified as the start of a plain list (in the right context) without needing to parse the entire element.
*** For
Avoiding parsing the entire element for an exit matcher would reduce redundant parses.
*** Against
This adds code complexity and introduces the potential for bugs.
How many elements can be reasonably early-detected? For example, src_org{#+begin_src foo} is not enough to detect the start of a source block because without the src_org{#+end_src} it is just plain text.
** Grab multiple characters in plaintext parser before checking exit matcher
Currently we check the exit matcher after each character inside the plain text parser (and many others). Are there character sequences we can assume no exit matcher will trigger between? For example, a contiguous string of latin-alphabet letters?
*** For
This could significantly reduce our calls to exit matchers.
*** Against
I think targets would break this.
The exit matchers are already implicitly building this behavior since they should all exit very early when the starting character is wrong. Putting this logic in a centralized place, far away from where those characters are actually going to be used, is unfortunate for readability.

View File

@@ -1,6 +1,7 @@
1. foo
1. plain-list
#+begin_center
#+end_center
2. bar
Is this still in the plain list?

View File

@@ -1 +0,0 @@
This folder is an investigation into whether or not my exit matchers should operate from the top down or bottom up.

View File

@@ -1 +0,0 @@
foo *bar baz * lorem* ipsum

View File

@@ -1,3 +0,0 @@
Looks like 2 blank lines always exits the top-level plain list.
Plain lists do not seem to go inside paragraphs but rather exist beside them.

View File

@@ -1,12 +0,0 @@
1. foo
bar
1. baz
lorem
ipsum
dolar

View File

@@ -1 +0,0 @@
Looks like table cells cannot contain lists but can contain bolds

View File

@@ -1,5 +0,0 @@
ip *su* m
| foo | bar |
|----------+-----|
| 1. lo *re* m | |

View File

@@ -0,0 +1,25 @@
#+BEGIN: clocktable :scope file :maxlevel 2
#+CAPTION: Clock summary at [2023-08-25 Fri 05:34]
| Headline | Time |
|--------------+--------|
| *Total time* | *0:00* |
#+END:
#+BEGIN: columnview :hlines 1 :id global
| ITEM | TODO | PRIORITY | TAGS |
|-------+------+----------+------------------------------|
| Foo | | B | |
|-------+------+----------+------------------------------|
| Bar | TODO | B | |
|-------+------+----------+------------------------------|
| Baz | | B | :thisisatag: |
| Lorem | | B | :thisshouldinheritfromabove: |
| Ipsum | | B | :multiple:tags: |
#+END:
* Foo
* TODO Bar
* Baz :thisisatag:
** Lorem :thisshouldinheritfromabove:
*** Ipsum :multiple:tags:
* Dolar ::
* cat :dog: bat

View File

@@ -0,0 +1,18 @@
#+begin_defun
foo
#+begin_lorem
,#+begin_center
bar
,#+end_center
ipsum
#+end_lorem
baz
#+end_defun
#+begin_center
#+begin_quote
#+begin_center
lorem
#+end_center
#+end_quote
#+end_center

View File

@@ -0,0 +1,12 @@
#+begin_defun
foo
#+begin_lorem
ipsum
#+end_lorem
bar
#+begin_center
#+begin_quote
baz
#+end_quote
#+end_center
#+end_defun

View File

@@ -0,0 +1,5 @@
#+begin_defun
foo
{{{bar(baz)}}}
#+end_defun

View File

@@ -0,0 +1,7 @@
1. foo
2.
bar
1.
#+begin_center
Still in the list
#+end_center

View File

@@ -0,0 +1,2 @@
- foo ::

View File

@@ -0,0 +1,11 @@
- foo :: bar
- cat ::
dog
- lorem
:: ipsum
-
lorem :: ipsum
- dolar *bold* foo :: ipsum
- big gap ::
stuff

View File

@@ -0,0 +1 @@
- {{{foo(bar)}}} :: baz

View File

@@ -0,0 +1,2 @@
- foo

View File

@@ -0,0 +1,9 @@
1. foo
1. bar
2. baz
2. lorem
ipsum

View File

@@ -0,0 +1,11 @@
1. foo
1. bar
2. baz
cat
2. lorem
ipsum

View File

@@ -0,0 +1,10 @@
1. cat
1. foo
1. bar
2. baz
2. lorem
ipsum

View File

@@ -0,0 +1,12 @@
1. cat
1. foo
1. bar
2. baz
2. lorem
2. dog
ipsum

View File

@@ -0,0 +1,7 @@
** foo
:PROPERTIES:
:DESCRIPTION: lorem
:ALT_TITLE: ipsum
:END:
bar

View File

@@ -0,0 +1 @@
%%(foo bar) ; baz

View File

@@ -0,0 +1,15 @@
#+name: foo
#+caption: bar
#+caption: baz
[[file:lorem/ipsum.png]]
#+name: cat
#+foo: dog
[[file:lorem/ipsum.png]]
#+name: cat
#+foo: dog
foo

View File

@@ -0,0 +1,22 @@
# Extra open
[cite/a/b-_/foo:unbalancedglobal[prefix;keyprefix @foo keysuffix;globalsuffix]
[cite/a/b-_/foo:globalprefix;unbalancedkey[prefix @foo keysuffix;globalsuffix]
[cite/a/b-_/foo:globalprefix;keyprefix @foo unbalancedkey[suffix;globalsuffix]
[cite/a/b-_/foo:globalprefix;keyprefix @foo keysuffix;unbalancedglobal[suffix]
# Extra close
[cite/a/b-_/foo:unbalancedglobal]prefix;keyprefix @foo keysuffix;globalsuffix]
[cite/a/b-_/foo:globalprefix;unbalancedkey]prefix @foo keysuffix;globalsuffix]
[cite/a/b-_/foo:globalprefix;keyprefix @foo unbalancedkey]suffix;globalsuffix]
[cite/a/b-_/foo:globalprefix;keyprefix @foo keysuffix;unbalancedglobal]suffix]
# balanced:
[cite/a/b-_/foo:gl[obalpref]ix;ke[ypref]ix @foo ke[ysuff]ix;gl[obalsuff]ix]

View File

@@ -0,0 +1,2 @@
[fn:2:This footnote [ has balanced ] brackets inside it]
[fn::This footnote does not have balanced [ brackets inside it]

View File

@@ -0,0 +1,6 @@
$foo
bar
baz
lorem
ipsum
dolar$

View File

@@ -0,0 +1,52 @@
non-link text
eww://
rmail://
mhe://
irc://
info://
gnus://
docview://
bibtex://
bbdb://
w3m://
doi://
file+sys://
file+emacs://
shell://
news://
mailto://
https://
http://
ftp://
help://
file://
elisp://
randomfakeprotocl://
non-link text
non-link text
eww:
rmail:
mhe:
irc:
info:
gnus:
docview:
bibtex:
bbdb:
w3m:
doi:
file+sys:
file+emacs:
shell:
news:
mailto:
https:
http:
ftp:
help:
file:
elisp:
randomfakeprotocl:
non-link text

View File

@@ -0,0 +1,3 @@
mailto:foo@bar.baz.
mailto:foo@bar.baz....

Some files were not shown because too many files have changed in this diff Show More