hello friends. i only recently discovered Pikchr after long thinking something like it needed to exist (and it turns out it does!). i particularly appreciate how easy it is to integrate into other tools.

as near as i can tell, however, none of the existing integrations or tools take the form of a pic(1)-like Unix filter/preprocessor that reads (for example) Markdown+Pikchr and writes Markdown+SVG for further processing by insert your favorite Markdown formatter. so i wrote one. :) it's super straightforward and nearly (but not entirely) trivial, so i'm surprised that one doesn't exist already. if i missed one out in the wild, please correct me.

https://github.com/zenomt/pikchr-cmd

in the spirit of pic(1), it searches its standard input for Pikchr start & end delimiters, transforming each delimited diagram into an inline SVG (inside a <div> to style the max-width), and copies everything else verbatim to the standard output. for simplicity, delimiters must be at the beginning of a line (not indented as allowed by Markdown) and have exactly three tildes or backticks. PIC delimiters (.PS and .PE) are also recognized.

it's just one C file to compile with pikchr.c, and requires POSIX regular expressions (that is, regcomp(3) and regexec(3)) which should already be in your standard library.

arrow right 225% "Markdown+Pikchr" "Source"
Tool: box rad 5px "pikchr" mono "Preprocessor" "(main.c)" mono fit
arrow same "Markdown+SVG" "Intermediate"
box same "Any Markdown" "Formatter" fit
arrow same "HTML+SVG" "Output"
arrow <-> down from Tool.s; "C API" ljust at 4pt right of last arrow
box with .n at last arrow.s same "Pikchr" "Formatter" "(pikchr.c)" mono fit

sadly i discovered that GitHub suppresses inline SVGs when browsing a repo's Markdown files, though they appear properly in GitHub Pages.

-michael thornburgh

https://github.com/zenomt/pikchr-cmd

Just FYI: a link to your project was added to the "external projects" list on the home page. If its description is wrong, please post a proposed replacement.

awesome, thanks! i hope folks find it useful.

If its description is wrong, please post a proposed replacement.

while the description is technically correct, i think it doesn't quite convey what i think is the most important aspect for a one-liner elevator pitch. instead how about:

pikchr-cmd, by Michael Thornburgh, is a command-line Pikchr preprocessor tool for use with any Markdown formatter.

(adjust capitalization as you like for Pikchr and Markdown :) )

... is a command-line Pikchr preprocessor tool for use with any Markdown formatter.

It's changed now.

(adjust capitalization as you like for Pikchr and Markdown :) )

Apparently the project's standard is to capitalize both. Good catch.

i've made a few additions to improve the tool's utility:

• start and end delimiters now consist of 3 or more consecutive tildes or backticks (though delimiters still need to start at the beginning of a line, so the tool doesn't need to implement a complete Markdown parser and so it's possible to include delimited Pikchr source in a code block)
• "svg-only" (no enclosing DIV with max-width) can be selected per-diagram with a start-delimiter modifier
• the "requote" modifier signals to include the original Pikchr source in an indented code block in the output (with or without the start and end delimiters according to the "delimiters" modifier) after the compiled SVG
• arbitrary other modifiers can be added in the start delimiter to be used as names/tags, and can be matched with the -N command-line option for extracting and compiling a specific diagram from the input document; this is useful for the GitHub case where an SVG can only be used in an external IMG reference but you'd still like the Pikchr source in the original document for plain-text consumption

(note: the open attribute on a <details> doesn't seem to work here in the forum so i didn't use it for this example)

There's an internal whitelist of attributes which are permitted, and that one is apparently not on the list. My quick attempt to add it revealed that i'll need to shift bitmasks of many enum elements to add a bit for it (in properly name-sorted order, in the middle of a longer bitmask list).

Anyway, this post is a reminder to self to fix that after some sleep.

Edit: that change is checked in to a branch now and is pending debate with the team on whether we want to squander our few remaining bitmask bits this way. (FWIW, it seems reasonable to me.)

that change is checked in to a branch now

looking at that check-in, while i see the definition of AMSK_OPEN, that doesn't seem to be added (or |ed) into the allowed attributes for details.

i see the definition of AMSK_OPEN, that doesn't seem to be added ...

Indeed, nice catch. How it works without that is not clear to me, but it does. It shouldn't, IMO.

That fix is checked in now.

How it works without that is not clear to me, but it does. It shouldn't, IMO.

that should be very concerning.

pending debate with the team on whether we want to squander our few remaining bitmask bits this way

it looks like the way the attribute checking works is: for each attribute, it's looked up with a binary search in a list with about 30 items (so about 4 strcmp()s on average per attribute) to see if it's even there, and if it is you have its bit, and you can then check if the bit is in the mask for the tag under consideration.

given the number of strcmp()s you're doing now and that the number of allowed attributes for almost all tags is small (except for img which allows 9), i bet it wouldn't be any more expensive, and also be more general, if you were to:

• have a function like my strword() to look for a "word" in a string (strstr(3) isn't quite what you want, because it would match if an attribute-under-consideration was a prefix of an allowed attribute)
• skip checking whether the attribute is one you have a bit for
• for each tag, have a simple string of allowed attributes separated by spaces
• an attribute is allowed if strword(allowedForTag, attr) answers not-NULL

while that last part would be O(N) in "number of allowed attributes" instead of O(log M) in "number of known attributes", it's probably a wash since N is usually like 3 or fewer, and M is 30.

it looks like the way the attribute checking works is ...

That lives in a corner of the tree we very rarely need to touch, so it's one of those "if it ain't broke, don't fix it" things. That said: patches improving it would be thoughtfully considered :).

it's one of those "if it ain't broke, don't fix it" things

it will be broken when it comes time to add a new attribute after one more after this one. at that time, an approach like i described above would make future maintenance much easier (adding a new attribute would be one word in one line per tag it needed to be allowed in) with, i expect, no measurable performance impact (which appears to have been a consideration given the current code).

patches improving it would be thoughtfully considered :)

since i don't run Fossil myself, i don't have either of "enough skin in the game" or "sufficient experience with the code to recognize whether i made things worse". and by just suggesting an algorithm without actually coding it myself (beyond an illustrative example of one possible implementation of one part), there's no IPR consideration for y'all or for me.

perhaps the time of "add a new attribute after one more after this one" will never come. but if it does, hopefully this thread will be remembered. :)