Fork me on GitHub

Scalateχ


0.3.7

@div(id:="my-div")
  @h1
    Hello World!
  @p
    I am @b{Cow}
    Hear me moo
    I weigh twice as much as you
    And I look good on the barbecue
<div id="my-div">
  <h1>Hello World!</h1>
  <p>
    I am <b>Cow</b>
    Hear me moo
    I weigh twice as much as you
    And I look good on the barbecue
  </p>
</div>

Scalatex is a language for generating rich HTML documents in Scala. It lets you DRY up your documents the same way you DRY your code. Unlike most templating languages, Scalatex is a thin shim on top of the Scala programming language. This means that some things which require built-in support or are otherwise impossible in other document/static-site-generators are completely trivial with Scalatex, for example:

All these things are trivial and elegant to perform with Scalatex, and difficult or messy to do with other static-site-generators. Although Scalatex generates HTML, you can easily build up whatever tags best describe what you want to do, and use those to build your document, rather than relying on the ad-hoc and messy set provided by browsers.

Scalatex lets you write your HTML in a hierarchical structure which then gets compiled to HTML. In Scalatex, everything is part of the output document, except for tokens marked with an @ sign. These correspond to HTML tags (provided by Scalatags), values you want to interpolate, control flow statements, function calls (or definitions!), basically anything that isn't plain text.

Scalatex is currently used to generate its own readme (here) as well as the e-book Hands-on Scala.js. It is only published for Scala 2.11.x for the time being.

Getting Started


To get started with Scalatex, add the following to your project/build.sbt:

addSbtPlugin("com.lihaoyi" % "scalatex-sbt-plugin" % "0.3.7")

And the following to your project in your build.sbt:

scalatex.SbtPlugin.projectSettings

scalaVersion := "2.11.4"

To begin with, let's create a file in src/main/scalatex/Hello.scalatex

@div
  Hello World

  @h1
    I am a cow!

Next, you can simply call

Hello().render

And it'll print out

<div>
  Hello World

  <h1>I am a cow!</h1>
</div>

There we have it, your first Scalatex document! You can put this on gh-pages, use it on a website, or where-ever you want.

What Scalatex Does


Scalatex converts every .scalatex file in its source folder into a corresponding Scala object. These objects have an apply method which returns a Scalatags Frag, which you can then call .render on to give you a HTML string. You can also do other things with the Scalatags Frag, but to learn more about it and Scalatags in general take a look at the Scalatags documentation.

Inside each Scalatex file, @-escaped things correspond to Scala keywords or names that are currently in scope. Apart from keywords, only scalatags.Text.all._ is imported by default. This brings in all the HTML tags that we used above to build those HTML fragments. However, you can also @import whatever other names you want, or refer to them fully qualified.

Exploring Scalatex


@div
  @h1("Hello World")
  @h2{I am Cow}
  @p
    Hear me @b{moo}
    I weigh twice as much as you
    And I look good on the barbecue
<div>
  <h1>Hello World</h1>
  <h2>I am Cow</h2>
  <p>
    Hear me <b>moo</b>
    I weigh twice as much as you
    And I look good on the barbecue
  </p>
</div>

Superficially, Scalatex does a small number of translations to convert the .scalatex document into Scala code:

This accounts for the variation in syntax that you see above. In general, you almost always want to use indentation-based blocks to delimit large sections of the document, only falling back to curly braces for one-line or one-word tags like @h2 or @b above.

Loops

@ul
  @for(x <- 0 until 3)
    @li
      lol @x
@ul
   @for(x <- 0 until 3){@li{lol @x}}
<ul>
  <li>lol 0</li>
  <li>lol 1</li>
  <li>lol 2</li>
</ul>

Scalatex supports for-loops, as shown above. Again, everything inside the parentheses are an arbitrary Scala expression. Here we can see it binding the x value, which is then used in the body of the loop as @x to splice it into the document.

In general, there are always two contexts to keep track of:

The value of strings or variables is completely identical in both contexts; it's only the syntax that differs.

Conditionals

@ul
  @for(x <- 0 until 5)
    @li
      @if(x % 2 == 0)
        @b{lol} @x
      @else
        @i{lol} @x
<ul>
  <li><b>lol</b> 0</li>
  <li><i>lol</i> 1</li>
  <li><b>lol</b> 2</li>
  <li><i>lol</i> 3</li>
  <li><b>lol</b> 4</li>
</ul>

Scalatex supports if-else statements, that behave as you'd expect. Here we're using one in conjunction with a loop to alternate the formatting of different items in a list.

Functions

@span
  The square root of 9.0
  is @math.sqrt(9.0)
<span>
  The square root of 9.0 is 3.0
</span>

Apart from splicing values into the document, you can also call functions, such as math.sqrt here.

Interpolations

@span
  1 + 2 is @(1 + 2)
<span>1 + 2 is 3</span>

You can also splice the result of arbitrary chunks of code within a Scalatex document. Using parens () lets you splice a single expression, whereas using curlies {} lets you splice a block which can contain multiple statements.

Blocks can span many lines:

@div
  1 + 2 is @{
    val x = "1"
    val y = "2"
    println(s"Adding $x and $y")
    x.toInt + y.toInt
  }
<div>
  1 + 2 is 3
</div>

External Definitions

You can use imports to bring things into scope, so you don't need to keep referring to them by their full name:

@import scala.math._
@ul
  @for(i <- -2 to 2)
    @li
      @abs(i)
<ul>
  <li>2</li>
  <li>1</li>
  <li>0</li>
  <li>1</li>
  <li>2</li>
</ul>

Since you can splice the value of any Scala expressions, of course you can splice the values that you defined yourself somewhere else:

object Stuff {
  object omg {
    def wrap(s: Frag*): Frag = Seq[Frag]("...", s, "...")
  }
  def str = "hear me moo"
}
@import Stuff._
@omg.wrap
  i @b{am} cow @str
...i <b>am</b> cow hear me moo...

Internal Definitions

You can also define things anywhere inside your Scalatex document. These definitions are only visible within the same document, and are scoped to any blocks they're defined within:

@object Foo{
  val const1 = 1337
}
@val const2 = 30000 + 1337

@p
  @def wrap(s: Frag*) = span(u(s))

  The first constant is @Foo.const1,
  the second is @const2, and wrapping
  looks like @wrap{hello @b{world}}
  or @wrap("hello ", b("world"))
<p>
  The first constant is 1337, the second
  is 31337, and wrapping looks like
  <span><u>hello <b>world</b></u></span> or
  <span><u>hello <b>world</b></u></span>
</p>

This is very useful for defining constants or functions which aren't needed anywhere else, but can help reduce repetition within the current document

Scalatex Site


The core of Scalatex is a very small, superficial translation from the .scalatex syntax to a Scalatags fragment, letting you spit out HTML strings. Nevertheless, there are other concerns common to most (all?) documents, and Scalatags provides a number of common utilities that can quickly turn your structured document into a pretty, browse-able web page.

Quick Start


If you want to get started quickly without fuss, given you've already added the SBT plugin to your build.sbt file, you should use scalatex.ScalatexReadme to get off the ground quickly:

lazy val readme = scalatex.ScalatexReadme(
  projectId = "readme",
  wd = file(""),
  url = "https://github.com/lihaoyi/scalatex/tree/master",
  source = "Readme"
)

This example usage would quickly create a project in the readme/ folder, that you can immediately work with. Simple place a Readme.scalatex file in readme/ and you can start writing Scalatex that will get turned into your documentation page. This default setup sets up all the commonly used functionality into the Main object, so make sure you start off with

@import Main._

to get access to functionality such as:

When all is said and done, run:

readme/run

To generate the Scalatex site inside the target/site folder.


The changes are, this will create a reasonable looking document with an interactive navigation bar, nice looking headers, etc.. If you wish to really customize Scalatex, look at the following sections to see how these various components of a Scalatex site actually work.

Section


One concern common to almost all documents is the idea of a section hierarchy: a tree of document sections where each section has a header and some contents, potentially containing more (smaller) sections. Scalatext provides the scalatex.site.Section helper that lets you do this easily:

@sect{Main Section}
  @p
    Hello World

  @sect{Subsection A}
    @p
      I am Cow
  @sect{Subsection B}
    @p
      Hear me moo
  @sect{Subsection C}
    @sect{Sub-sub-section C1}
      @p
        I weigh twice as much as you
    @sect{Sub-sub-section C2}
      @p
        And I look good on
        the barbecue

Main Section


Hello World

Subsection A


I am Cow

Subsection B


Hear me moo

Subsection C


Sub-sub-section C1

I weigh twice as much as you

Sub-sub-section C2

And I look good on the barbecue


As you can see, all the indented sections are automatically set to smaller headings. This is of course doable manually with @h1 @h2 @h3 tags etc., but then you have to manually keep track of how deep something is. With Section, this is kept track of for you.

You can override the default rendering of Sections to e.g. use different headers:

@import scalatex.site.Section
@object sect extends Section{
  override val headers: Seq[Header] =
    Seq(b, u, s)
}

Plans for the Revolution

Hello World

Coup d'etat

I am Cow

New World Order
Monetary Policy

I weigh twice as much as you

Conscription

And I look good on the barbecue


Above, you see it using bold, underlines and strike-throughs to delimit the boundaries between headers. You can also override things in more detail using the Section.Header class instead of simple tags.

You can use @sect.ref to link to another part of the document:

@sect{Page with Refs}
  @p
    Hello World, look at @sect.ref{Sect A}

  @sect{Sect A}
    @p
      I am Cow
  @sect{Sect B}
    @p
      Hear me moo. Click
      @sect.ref{Page with Refs} to
      go back to the top.


@sect.structure.toString
@br
@br
@sect.usedRefs.toString

Page with Refs


Hello World, look at Sect A

Sect A


I am Cow

Sect B


Hear me moo. Click Page with Refs to go back to the top.

Tree(root,ArrayBuffer(Tree(Page with Refs,ArrayBuffer(Tree(Sect A,ArrayBuffer()), Tree(Sect B,ArrayBuffer())))))

Set(Page with Refs, Sect A)

Any links created with @sect.ref are stored in an @sect.usedRefs property as a list. We also expose a @sect.structure property which allows you to inspect the tree of sections. If any of the sections referred to by sect.ref are not found, site-generation will fail with a helpful error telling you which one so you can fix it.

Here we're just stringifying the Tree structure and placing it as text on the page, but you can easily recurse over the Tree structure and use it to e.g. build a table of contents, or a navigation bar.

Highlighter


Scalatex Site provides a Highlighter class, which lets you easily highlight snippets of code pulled from files on disk. First, you need to instantiate a Highlighter object:

package scalatex.highlighter

object hl extends scalatex.site.Highlighter

Once you have done so, you can highlight individual code snippets:

@import Main._
@div
  @hl.xml{<div><b>Some</b> HTML</div>}
@div
  @hl.js("f = function(x){return x+1}")
@div
  @hl.scala
    val f = (x) => { x + 1 }

<div><b>Some</b> HTML</div>
f = function(x){return x+1}
val f = (x) => { x + 1 }

Or you can use it to reference files in your code base in whole; e.g. here is Scalatex's plugins file:

@import Main._
@hl.ref(wd/'project/"build.sbt")

unmanagedSources in Compile ++= {
  val root = baseDirectory.value.getParentFile
  (root / "scalatexSbtPlugin" ** "*.scala").get
}

addSbtPlugin("com.jsuereth" % "sbt-pgp" % "1.0.0")

addSbtPlugin("org.scala-js" % "sbt-scalajs" % "0.6.13")

Note here we are using Ammonite to construct a path to pass to @hl.ref; this path takes a working directory which you'll need to define yourself (here imported from Main) via val wd = ammonite.ops.cwd.

Apart than referencing the whole file, you can also grab text from the line containing a key-string to the line containing another; here is parts of Scalatex's build file, from lazy val api to the next lazy val:

@import Main._

@hl.ref(
  wd/"build.sbt",
  start = "lazy val api",
  end = "lazy val"
)

lazy val api = project.settings(sharedSettings:_*)
  .settings(
    name := "scalatex-api",
    libraryDependencies ++= Seq(
      "com.lihaoyi" %% "utest" % "0.4.4" % "test",
      "com.lihaoyi" %% "scalaparse" % "0.4.2",
      "com.lihaoyi" %% "scalatags" % Version.scalaTags,
      "org.scala-lang" % "scala-reflect" % scalaVersion.value
    ),
    testFrameworks += new TestFramework("utest.runner.Framework")
  )

Using multiple key-strings to navigate to a particular start- or end- line; here is the body of the unit test scalatex.BasicTests.parenArgumentLists.attributes, only from the check to the } closing that test.

@import Main._
@import ammonite.ops._
@hl.ref(
  'api/'src/'test/'scala/
  'scalatex/"BasicTests.scala",
  start = Seq(
    "parenArgumentLists",
    "attributes", "check"
  ),
  end = Seq("  }")
)

check(
  tw("""
    @div(id:="my-id"){ omg }
    @div(id:="my-id")
      omg
  """),
  """
    <div id="my-id">omg</div>
    <div id="my-id">omg</div>
  """
)

Note the spaces before the }, which ensures that it doesn't grab the earlier } in the string-literal which is part of the test body.

Normally, the language used to highlight is inferred from the suffixMappings you defined when instantiating the Highlighter. You can override it manually by passing it in:

@import Main._

@def example(s: Seq[String],
             cls: String) = {
  val path =
    wd/'api/'src/'test/'scala/
    'scalatex/"BasicTests.scala"
  hl.ref(
    path, s, Seq("\"\"\""), cls
  )
}

@p
  Compiling:

@example(Seq("parenArgumentLists", "attributes", "@div"), "scala")

@p
  Gives you:

@example(Seq("parenArgumentLists", "attributes", "<div"), "html")

Compiling:

@div(id:="my-id"){ omg }
@div(id:="my-id")
  omg

Gives you:

<div id="my-id">omg</div>
<div id="my-id">omg</div>

As you can see, this is especially useful from grabbing examples from unit tests. You can encapsulate the boilerplate in a function, and easily grab both the code and the expected output from the unit tests highlighted in their own particular way to include in the document.

Lastly, you can define a pathMappings property, which is a sequence of prefixes which the highlighter tries to match against the files you're referencing. If any of them match, a link is generated (in the top right corner of the code snippet) containing the mapped path prefix combined with the remainder of the file path. Here's one that links everything to github:

@import Main._
@object hl extends scalatex.site.Highlighter{
  override def pathMappings = Seq(
    wd -> (
      "https://github.com/lihaoyi/" +
      "Scalatex/blob/master/"
    )
  )
}

@hl.ref(
  wd/'api/'src/'test/'scala/
  'scalatex/"BasicTests.scala",
  start = Seq(
    "parenArgumentLists",
    "attributes", "check"
  ),
  end = Seq("  }")
)

check(
  tw("""
    @div(id:="my-id"){ omg }
    @div(id:="my-id")
      omg
  """),
  """
    <div id="my-id">omg</div>
    <div id="my-id">omg</div>
  """
)

You can customize the link to link to other places (bitbucket, local files, etc.) by changing the mapped URL. You can also pass in multiple pathMappings have links to files in different folders map to different base URLs, e.g. if you are referencing files in sub-repositories that have different public code


In general, Highlighter.ref is very helpful to quickly grab snippets of code from your unit tests, examples, or other places to include in your document. Using Highlighter.ref, you should never need to copy & paste code from unit tests to your documentation: simply refer to it directly, and you are guaranteed that your documentation will always be compile-able and in sync with what your code actually does in its unit tests.

lnk


Scalatex provides the lnk function. This can be called with a single URL:

@lnk("http://www.google.com")

Or with a custom name

@lnk("Google", "http://www.google.com")

And generates a link to that URL. @lnk also keeps track of all the links you create, and you can validate them later by running sbt "readme/run --validate" on your Scalatex readme project. This will reach out to every link you use and fail with an error if any of them are broken.

Site


Finally, if all you want is to produce a working, pretty website without having to set up CSS and hosting and all that yourself, Scalatex provides a Site trait which lets you do this easily:

val site = new scalatex.site.Site {
  def content = Map("index.html" -> (defaultHeader, Hello()))
}
site.renderTo(wd/'site/'target/'output)

This will result in a single HTML file index.html in the site/target/output/ folder, as well as the JavaScript for Highlight.js and CSS for Font Awesome and Less CSS bundled together into scripts.js and styles.css. From there, you can simply copy the whole folder and serve it on whatever static file hosting (Github Pages, S3, etc.).

Although Site provides reasonable defaults for all these things they are entirely customizable, and you just need to override various definitions when instantiating the Site. For example, here's a slightly more complex Site that generates multiple HTML pages (page1.html and page2.html) and generates them with custom names for the CSS and JavaScript bundles.

val site = new scalatex.site.Site {
  override def scriptName = "custom.js"
  override def stylesName = "custom.css"
  override def defaultHeader = super.defaultHeader ++ Seq(
    script("console.log('Hi!')")
  )

  def content = Map(
    "page1.html" -> (defaultHeader, Hello()),
    "page2.html" -> (defaultHeader, About())
  )
}
site.renderTo(wd/'site/'target/'output2)

Similarly, you can choose to supplement the default settings with your own customization. The following example specifies a different title for each of the two pages.

val site = new scalatex.site.Site {
  val mooMarker = "Moooo"
  val bbqMarker = "Barbecue!"

  def content = Map(
    "page1.html" -> (defaultHeader ++ Seq(scalatags.Text.tags2.title(mooMarker)), Hello()),
    "page2.html" -> (defaultHeader ++ Seq(scalatags.Text.tags2.title(bbqMarker)), About())
  )
}
site.renderTo(wd / 'site / 'target / 'output)

There are several other things you to can override, for a full list, look at the source code of Site to see what's available.

Advanced Techniques


This section goes through a number of things which are not part of Scalatex by default, but are easily doable by extending Scalatex with snippets of Scala code. In general, all of these are probably sufficiently ad-hoc that the exact implementation would vary from project to project and so a built-in implementation wouldn't make sense.

Custom Tags

Since tags are just functions, it is easy to define custom tags just by defining your own function that returns a Scalatags fragment. For example, auto-linking URL-looking-text isn't built in as part of Scalatex, but here's a custom tag that adds the capability:

@def autoLink(url: String) = {
  if(url.contains("://"))
    a(url, href:=url)
  else
    a(url, href:=s"http://$url")
}

@p
  I always like going to either
  @autoLink{www.google.com} or
  @autoLink{https://www.facebook.com}
  in order to look for things.

I always like going to either www.google.com or https://www.facebook.com in order to look for things.


Again, if you are using the @def in multiple files you can define it in Scala code and import it, but if you're only using it in one .scalatex file you can simply define it locally.

Note how we check for the :// inside the URL and default it to http:// if it's missing; this is just plain-old-Scala, since the tag is just a Scala function.

Generating a Table of Contents

Generating a TOC has also traditionally been a pain point for documents. There are two common scenarios:

With Scalatex, generating a TOC from the document is both easy and customizable. Here is a trivial example:

@import scalatex.site.Section
@import scalatex.site.Tree
@object sect extends Section()

@sect{TOC Main Section}
  @p
    Hello World

  @sect{TOC Subsection A}
    @p
      I am Cow
  @sect{TOC Subsection B}
    @p
      Hear me moo
  @sect{TOC Subsection C}
    @sect{TOC Sub-sub-section C1}
      @p
        I weigh twice as much as you
    @sect{TOC Sub-sub-section C2}
      @p
        And I look good on
        the barbecue

@hr
@b{Table of Contents}
@{
  def rec(t: Tree[String]): Frag = {
    val id =
      "#" + sect.munge(t.value)
    div(
      a(t.value, href:=id),
      ul(t.children.map(
        c => li(rec(c))
      ))
    )
  }
  sect.structure.children.map(rec)
}

TOC Main Section


Hello World

TOC Subsection A


I am Cow

TOC Subsection B


Hear me moo

TOC Subsection C


TOC Sub-sub-section C1

I weigh twice as much as you

TOC Sub-sub-section C2

And I look good on the barbecue


Table of Contents

As you can see, one short recursive function and is all that is necessary to transform the captured structure into a ul tree, with links to all the section headers! If you wish you can trivially modify this function to e.g. generate different HTML/CSS, only show top-level or top-and-second-level headers, or all manner of other things.

Note the use of the sect.munge function. This is the function that converts the headings (with their spaces and other special characters) into a HTML element ID that can be used for the anchor link. By default, it simply removes spaces, but you can override it in the Highlighter object if you want some custom behavior.

Why Scalatex


Scalatex requires that you learn a new syntax for doing things, which is incompatible with Markdown, Textile, and any other markup language you previously learned. Apart from that, Scalatex itself was the culmination of quite a lot of work to implement. Why did I do this?

In general, Scalatex aims to make eliminating repetition and C&P sufficiently easy and elegant. Most template processors (with the exception of Twirl) start from text, and apply a series of transformations that turn the document into the HTML that you want. This ends up implementing a custom programming language with a custom syntax right inside your templates, often with

This is not great, but it always happens given a sufficiently complex document. For example, here's a sample of "real world" markdown taken from the Scala.js website, the render a relatively simple page:

    
      ---
      layout: page
      title: Passing basic types to JavaScript
      ---
      {% include JB/setup %}

      ## Problem

      You need to pass simple types (String, Int, Boolean) to
      JavaScript functions.

      ## Solution

      Let us illustrate the case by writing some functions on the host-page.

      Edit `index-dev.html` and add the following function:

      {% highlight scala %}
      function happyNew(name, year) {
        alert("Hey " + name + ", happy new " + year + "!");
      }
      {% endhighlight %}

      As we saw in the recipe of [accessing the global scope][1],
      we need to make the following import:

      {% highlight scala %}
      import scala.scalajs.js
      import js.Dynamic.{ global => g }
      {% endhighlight %}

      Now on the Scala side you can call it like this:

      {% highlight scala %}
      def main(): Unit = {
        g.happyNew("Martin", 2014)
      }
      {% endhighlight %}

      Which will result in the values being used and the corresponding
      alert being shown.

      [1]: ./accessing-global-scope.html

Note how we have:

This is a markdown document, but the same fate ends up befalling any complex document written in any existing document generation system: you end up with a chimera of 3 different languages spliced together using regexes. Scalatex fixes this not by making custom-logic unnecessary, but by making custom-logic so easy and standardized such that you never need to fall back to hacky regex manipulation.

Furthermore, by leveraging the Scala compiler, Scalatex also provides many things that other markup languages do note:

These are things that virtually no markup generators provide, but Scalatex does. Here are a few more detailed comparisons between Scalatex and alternatives.

Why Not: Markdown


Markdown is a lightweight format for writing rich text that has gained popularity recently. It has advantages of being extremely readable in plain-text: Markdown documents are formatted often exactly as someone would format emails or chat.

Markdown has the advantages of being:

However, it also has some downsides:

In general, Markdown is far more concise than Scalatex, and probably better for writing small documents like comments or chat boxes. For anything larger, you probably do want to define larger, re-usable components to deal with, rather than working directly with the fixed set of primitives markdown provides. Instead of working with fenced-code-snippets, links, and headers, you're working with references to your unit test code, or document sections that automatically provide the correct sized headers.

This makes large-scale documents much more pleasant, to the extent that you can perform large-scale refactorings of your Scalatex document without worrying about broken intra-document links or a stale table-of-contents.

Why Not: Twirl


Twirl is the templating engine used by the Play framework. It provides a Scala-based syntax (and implementation!) very similar to Scalatex.

In general, Scalatex is heavily inspired by Twirl: the original implementation started off as a fork of the Twirl source code, and though it has now diverged, the similarities are obvious. Scalatex has attempted to clean up a lot of unnecessary or ad-hoc cruft in the Twirl language, in an attempt to make something cleaner and nicer to use. For example:

Scalatex is still missing a few features:

In general, Scalatex aims to be much more regular and lighter weight than Twirl. You probably wouldn't want to use Twirl to write a book due to its excess of curlies and XML, whereas with Scalatex you can (and I have). At the same time, Scalatex keeps the best part of Twirl - the abilities to eliminate repetition using plain Scala - and extends to to work more regularly.

The limitations are not fundamental, but purely a matter of how Scalatex is being used now: static documents, rather than dynamic pages. If there's interest removing them would not be hard.

Changelog


0.3.7


0.3.6


0.3.5


0.3.4


0.3.3


0.3.2