Next steps for KumaScript

Notes from MDN London meeting, September 2018.

Problems with current KumaScript

  • Untested code that is used by thousands of pages
  • No comments, documentation or specification for macros
  • Too many macros
  • Too much duplication of code
  • Bad organization (“module” macros, lumped in data, l10n)
  • L10n of text means editing code, which can lead to fall out
  • No portability (e.g. compat table rendering outside of KS)
  • A one line change in a “data macro” can cause fall out
  • The repository has no clear maintainer, everyone maintaining it often means no one maintaining it (see the number of current open PRs)
  • The KumaScript codebase is outdated, uses ejs and other aged node modules.

Decisions taken

Based on the problems outlined above and a discussion about the history of KumaScript, we then decided what we could do to address some of these problems. Here are some of the things will we do in the short term:

  • Broadly delete macros that are of little use.
    • With fewer macros, everything should become easier:
      • easier to write tests
      • easier to reason about
      • easier to refactor
      • easier to replace them with better solutions.
  • Analyze KS landscape
    • Classify macros to help us understand what we use macros for now, and make recommendations for what we should do with whole categories of macros.
  • Write and enforce policies for macros
    • Once we’ve deleted many macros and analysed the landscape, it should become clear which kinds of macros we should keep. We will derive policies from these macros and make clear what we allow and disallow in KumaScript.
  • Testing is mandatory for code changes
    • If you want to make code change to a macro, and the macro doesn’t have tests, then you should add basic tests in your PR. If it is a macro that is used on thousands of pages, this is most important.
  • De-macrofiy or delete inactive MDN content
    • In London we’ve successfully removed all macros from the Archive and replaced them with their rendered version. This was done automatically using the PUT API (which is only available to MDN staff).
    • We will continue de-macrofying other parts of MDN that are not Web docs and outdated (XUL, XPCOM, etc).
    • We might delete content that is very out of date and not used by anyone anymore.

Work in progress

  • Macro deletion spreadsheet. We’re working on broadly deleting useless macros. You’re welcome to help, however deleting macros can be tricky, so this requires some digging and deep understanding of the fragile KumaScript code base. Coordinate with @wbamberg and @fscholz before doing any work on this.
  • As of the London meeting, we started with 432 macros we were quickly able to get down to 366 after the work week. See https://developer.mozilla.org/en-US/dashboards/macros for the current list of macros and their usage.
  • Will and Florian are going to work on presenting a “KumaScript landscape” overview and will try to come up with a macro policy. Macro deletion can continue meanwhile.

KumaScript in 2019

  • We will continue our short term cleanup efforts and then make a call sometime in 2019 if we should attempt to get rid of KumaScript altogether or try to modernize it for future use.
3 Likes

I’d like to point to my indicator style class simplification project, which I believe is relevant to this:

Will and Florian are going to work on presenting a “KumaScript landscape” overview

This doc is here:

1 Like

I have a number of thoughts on that plan, although generally it’s a good start. There are a couple of points where I strongly disagree, but I’m not at this very moment feeling coherent enough to write it all down adequately. :slight_smile:

1 Like

I’ve been putting together an experimental future syntax for replacing “One Page” macros at:

Hm. I guess I’m not picking up what the idea is here…

The goal is to use a Liquid‑esque syntax for dynamic page generation using in‑page scripting syntax embedded inside the {% and %} delimiters.

Anyway, here’s a “test” page which shows how the WebExtAllCompatTables macro could be replaced:

Anyway, I’m still fleshing out the details of the syntax and which functionality we’d want to have.

The obvious problem with in-page scripting like that is the potential security and stability concerns. We would really need the ability to lock down pages to only be editable by trusted individuals or groups before we could use a capability like that.

1 Like

That and the principle of least power, which is why I want to use a Liquid‑like1 syntax, rather than full blown EJS, like the old /docs/Template: namespace.


Footnote 1
MediaWiki uses a similarly limited template syntax which anyone can use to create templates.

Can the sharing settings please be modified so that comments can be made directly on the document like in the macro massacre spreadsheet?

This way, I don’t have to post in this thread every time I need to comment on something.

I don’t really want to make the doc globally commentable. Can you comment in this thread?

I have made some comments to the macros spreadsheet which I’d like to see addressed:

I’d like to be assigned to moving GroupData and InterfaceData to mdn‑data, as I’m doing this in:

1 Like