Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for manpages via mandoc #1196

Closed
wants to merge 22 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,9 @@ git:
before_install:
- sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv 379CE192D401AB61
- echo "deb https://dl.bintray.com/nxadm/rakudo-pkg-debs `lsb_release -cs` main" | sudo tee -a /etc/apt/sources.list.d/rakudo-pkg.list
- sudo add-apt-repository ppa:jordemort/mandoc -y
- sudo apt-get update -qq
- sudo apt-get install perl rakudo-pkg
- sudo apt-get install mandoc perl rakudo-pkg xmlstarlet
- export PATH=$PATH:/.perl6/bin:/opt/rakudo-pkg/bin
- curl -L http://cpanmin.us | perl - --sudo App::cpanminus
- sudo cpanm --installdeps --notest Pod::Simple
Expand Down
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ you wish to run the library. You can also run `script/bootstrap` to fetch them a
* [.asciidoc, .adoc, .asc](http://asciidoc.org/) -- `gem install asciidoctor` (http://asciidoctor.org)
* [.pod](http://search.cpan.org/dist/perl/pod/perlpod.pod) -- `Pod::Simple::XHTML`
comes with Perl >= 5.10. Lower versions should install Pod::Simple from CPAN.
* [.mdoc](http://mandoc.bsd.lv/man/mdoc.7.html), [.man](http://mandoc.bsd.lv/man/man.7.html) -- `apt-get` or `brew install mandoc xmlstarlet` (http://mandoc.bsd.lv/ http://xmlstar.sourceforge.net/)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this suggested change has broken diff.


Installation
-----------
Expand Down
11 changes: 11 additions & 0 deletions lib/github/commands/man2html
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -e

# process it, and print everything from the first <h1> to </body>
mandoc -Thtml -Ofragment "$@" \
| xml -q tr --html --omit-decl "$(dirname "$0")/man2html.xslt" - \
| awk '
/<h1/ { flag = 1 }
/<\/body>/ { flag = 0 }
flag
'
114 changes: 114 additions & 0 deletions lib/github/commands/man2html.xslt
Original file line number Diff line number Diff line change
@@ -0,0 +1,114 @@
<?xml version="1.0" encoding="utf-8"?>
<xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">

<!-- Default: match anything and apply templates to it -->
<xsl:template match="node()|@*">
<xsl:copy>
<xsl:apply-templates select="node()|@*"/>
</xsl:copy>
</xsl:template>

<!-- Strip all the attributes off of these -->
<xsl:template match="h1">
<h1><xsl:apply-templates /></h1>
</xsl:template>

<xsl:template match="h2">
<h2><xsl:apply-templates /></h2>
</xsl:template>

<xsl:template match="h3">
<h3><xsl:apply-templates /></h3>
</xsl:template>

<!-- Turn these into paragraphs -->
<xsl:template match="div[@class='Pp']">
<p><xsl:apply-templates /></p>
Copy link

@ischwarze ischwarze Nov 11, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This replacement is WRONG, you can't do that. In mandoc HTML output, paragraphs often contain flow content, but HTML only permits phrasing content inside <p> elements. So what you are doing here is introducing HTML syntax errors.

Copy link

@Alhadis Alhadis Jan 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ischwarze You aren't wrong, but mistakes like these really, really, really common in GitHub's front-end code, which is yet another giant bowl of "tag-soup" for their designers to play in. Nobody's gonna notice this syntax violation when the entire site is a scrapbook of "do whatever, as long as it looks good". :(

</xsl:template>

<!-- Indent these -->
<xsl:template match="div[@class='Bd-indent' or @class='Bd Bd-indent']">
<ul><xsl:apply-templates /></ul>
</xsl:template>

<!-- Headers don't need to be linking to themselves -->
<xsl:template match="a[@class='permalink']">
<xsl:apply-templates />
</xsl:template>

<!-- SHOUTING_SNAKE_CASE → calm-kebab-case -->
<xsl:template match="a[@class='Sx']">
<xsl:variable name="href" select="
translate(@href, 'ABCDEFGHIJKLMNOPQRSTUVWXYZ_',
'abcdefghijklmnopqrstuvwxyz-')" />
<a href="{$href}"><xsl:apply-templates select="node()" /></a>
</xsl:template>

<!-- Pull the manual-text div out -->
<xsl:template match="div[@class='manual-text']">
<xsl:apply-templates />
</xsl:template>

<!-- These were display: inline, so just give the content -->
<xsl:template match="div[@class='Nd' or @class='Bf' or @class='Op']">
<xsl:apply-templates />
</xsl:template>

<!-- These were display: italic -->
<xsl:template match="span[@class='Pa' or @class='Ad']">
<samp><xsl:apply-templates /></samp>
</xsl:template>

<!-- These were font-weight: bold -->
<xsl:template match="span[@class='Ms']">
<b><xsl:apply-templates /></b>
</xsl:template>

<xsl:template match="dl[@class='BL-diag']/dt">
<b><xsl:apply-templates /></b>
</xsl:template>

<xsl:template match="code[@class='Nm' or @class='Fl' or @class='Cm' or @class='Ic' or @class='In' or @class='Fd' or @class='Fn' or @class='Cd']">
<b><code><xsl:apply-templates /></code></b>
</xsl:template>

<!-- Monospace -->
<xsl:template match="var[@class='Ar']">
<code><var><xsl:apply-templates /></var></code>
</xsl:template>

<xsl:template match="a[@class='Xr']">
<code><xsl:apply-templates /></code>
</xsl:template>

<!-- Remove header table -->
<xsl:template match="table[@class='head']"/>

<!-- Reformat footer table and pull some header stuff into it -->
<xsl:template match="table[@class='foot']">
<hr/>
<table>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='head-ltitle']"/>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='head-vol']"/>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='foot-os']"/>
<xsl:apply-templates select="//td[text() and text() != '()' and @class='foot-date']"/>
</table>
</xsl:template>

<!-- Turn head header/footer cells into rows -->
<xsl:template match="td[@class='head-ltitle']">
<tr><th>Title:</th><td><xsl:apply-templates /></td></tr>
</xsl:template>

<xsl:template match="td[@class='head-vol']">
<tr><th>Volume:</th><td><xsl:apply-templates /></td></tr>
</xsl:template>

<xsl:template match="td[@class='foot-date']">
<tr><th>Date:</th><td><xsl:apply-templates /></td></tr>
</xsl:template>

<xsl:template match="td[@class='foot-os']">
<tr><th>Manual:</th><td><xsl:apply-templates /></td></tr>
</xsl:template>
</xsl:stylesheet>
1 change: 1 addition & 0 deletions lib/github/markup.rb
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ module Markups
MARKUP_RDOC = :rdoc
MARKUP_RST = :rst
MARKUP_TEXTILE = :textile
MARKUP_MANPAGE = :manpage
MARKUP_POD6 = :pod6
end

Expand Down
2 changes: 2 additions & 0 deletions lib/github/markups.rb
Original file line number Diff line number Diff line change
Expand Up @@ -57,3 +57,5 @@

command(::GitHub::Markups::MARKUP_POD6, :pod62html, /pod6/, ["Pod 6"], "pod6")
command(::GitHub::Markups::MARKUP_POD, :pod2html, /pod/, ["Pod"], "pod")

command(::GitHub::Markups::MARKUP_MANPAGE, :man2html, /(?:[1-9](?![0-9])[a-z_0-9]*|0p|n|man|mdoc)(?:\.in)?/, ["Roff Manpage"], "manpage")
2 changes: 1 addition & 1 deletion script/bootstrap
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash

set -e

Expand Down
2 changes: 1 addition & 1 deletion script/bootstrap.contrib
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash

set -e

Expand Down
2 changes: 1 addition & 1 deletion script/cibuild
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
#!/bin/bash
#!/usr/bin/env bash

set -e

Expand Down
8 changes: 4 additions & 4 deletions test/markup_test.rb
Original file line number Diff line number Diff line change
Expand Up @@ -66,10 +66,10 @@ def call
f.close_write
f.read
end
assert_html_equal expected, actual, <<message
#{File.basename expected_file}'s contents are not html equal to output:
#{diff}
message
assert_html_equal expected, actual, <<~message
#{File.basename expected_file}'s contents are not html equal to output:
#{diff}
message
end
end

Expand Down
6 changes: 6 additions & 0 deletions test/markups/README.man
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.Dd $Mdocdate: March 23 1981 $
.Dt README 1
.Os GitHub
.Sh README.man

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an mdoc(7) syntax error, which is probably not a good thing to have in an example/test file. The first section header in a manual page MUST be exactly ".Sh NAME". Your line is also invalid because it contains lower case letters.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And after that it should provide data for whatis(1)'s scraping; see lexgrog(1)'s WHATIS PARSING as an example.

.Sh NAME
.Nm foo
.Nd program to do something

.Nm README
.Nd This is an example readme in mandoc

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is an invalid example for a second reason. In addition to the NAME section, at least the DESCRIPTION section is also required in a manual page. So you should probably add

.SH DESCRIPTION
And one line of text.

or something like that.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ingo probably means .Sh

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, yes, of course, ".Sh DESCRIPTION", thanks for noticing.

23 changes: 23 additions & 0 deletions test/markups/README.man.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
<h1>README.man</h1>
<p><b><code>README</code></b> — This is
an example readme in mandoc</p>

<hr>
<table>
<tr>
<th>Title:</th>
<td>README(1)</td>
</tr>
<tr>
<th>Volume:</th>
<td>General Commands Manual</td>
</tr>
<tr>
<th>Manual:</th>
<td>GitHub</td>
</tr>
<tr>
<th>Date:</th>
<td>March 23, 1981</td>
</tr>
</table>