DefaultAPISidebar, APIRef and GroupData

exe-boss · May 10, 2019, 10:45pm

I personally recommend APIOverviewSidebar, since StandardOverviewSidebar could be confused with other namespaces.

wbamberg · May 12, 2019, 9:37pm

I personally recommend APIOverviewSidebar , since StandardOverviewSidebar could be confused with other namespaces.

Yeah, I think so too. I’m a bit sceptical that renaming will help much though, tbh. I did find this: https://wiki.mozilla.org/MDN/Archives/Meetings/Community/2015/09-23#News which is helpful especially in explaining “Default”:

This work well for all simple API, that have a couple of guides and some reference

For large APIs, we will still need to build more complex sidebars, like for WebRTC: https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API

This is another of my gripes, which I will eventually get to, that several APIs have their own handcrafted sidebar macros, and I think we should aim to converge on one* sidebar implementation.

* (or at least < 30 )

Edited to add: I’m very tempted to suggest that instead of talking about “APIs” and “Groups”, we talk about “Specifications” throughout, since that seems like a more accurate definition of these things. It’s telling that the section listing all these things in Web/API is called “Specifications”: https://developer.mozilla.org/en-US/docs/Web/API#Specifications.

Among other things, that could mean renaming DefaultAPISidebar -> SpecificationSidebar.

exe-boss · May 11, 2019, 12:48am

I’ve got a few plans for this, including adding sidebar generation and handling shared code to the mdn namespace:

(needs to be rebased, depends on mdn/kumascript#1107)

wbamberg · May 12, 2019, 3:40pm

Guides get listed twice

DefaultAPISidebar lists guides in the sidebar. It takes guides from subpages of the overview page and from the guides listed in the group. If a single guide appears in both places, it gets listed twice in the sidebar.

See e.g.

This seems like an easy fix: have DefaultAPISidebar check the URLs and remove duplicates.

Unfortunately it isn’t, because of page moves and redirects. Suppose a guide page is initially created outside the API overview page, and is listed in GroupData under that URL. Then we move it under the API overview page, it will appear twice and a simple check of the URLs will not show that. Perhaps page JSON can tell us about redirects? - but still this doesn’t look like a good direction.

My first preference here was to fix this by removing the “guides” property of GroupData and only looking for guides as subpages of the API overview page. That seems simpler, and a rule that guides should live under the API overview page seems good generally for documentation coherence.

However there are a couple of problems: the obvious one is that sometimes there might be a good reason for guides to live outside the API subtree. Perhaps we should give writers that freedom. The less obvious one (which is more convincing to me) is that writers probably want to order the guides in the sidebar, like:

getting started with the foo API

advanced uses of the foo API

some really niche stuff about the foo API

The WebVR API sidebar is quite a good example of that. The first half of the sidebar lists guides as they are returned by subpages, and it doesn’t make sense:

Using VR controllers with WebVR

Using the WebVR API

WebVR concepts

The second half has the same guides in the order they come in GroupData, and does make sense:

Using the WebVR API

WebVR concepts

Using VR controllers with WebVR

So I think we should change the macro so it doesn’t look in API overview subpages. Correspondingly, we’ll need to add “implicit” guide pages to GroupData.

exe-boss · May 12, 2019, 11:06pm

One option would be to have guides in GropData override subpage guides, which would be placed after the last subpage guide in GroupData or the beginning, if GroupData only contains non-overview subpage guides.

Option two would be to only do overview subpage guides if the GroupData guides key is missing or doesn’t contain subpages of the overview page.

Also, $json should support returning redirect metadata, which would help hide Firefox OS APIs from the API sidebar.

wbamberg · May 13, 2019, 3:33am

One option would be to have guides in GropData override subpage guides, which would be placed after the last subpage guide in GroupData or the beginning, if GroupData only contains non-overview subpage guides.

Sorry, I don’t understand this.

Option two would be to only do overview subpage guides if the GroupData guides key is missing or doesn’t contain subpages of the overview page.

We could do this. But as I said above, I think it’s inappropriate for guides to be taken from subpages, since guides typically want to be ordered. In this respect guides are different from something like a list of methods.

Also, $json should support returning redirect metadata, which would help hide Firefox OS APIs from the API sidebar.

Yes, as I said above, whether or not it’s possible, I think this is the wrong way to go. I would prefer us to be trying to simplify this situation, not to make it more complex.

chrisdavidmills · May 13, 2019, 9:12am

Thanks for doing all this work Will, to make sense of this mess. I must admit to being confused about the difference between these two macros for years. The wrong usage of them is probably largely my fault.

I’d be happy to have a name change to make them less confusing. APIRef is used in too many places to change the name, so renaming DefaultAPISidebar to SpecificationSidebar sounds good. Or maybe APILandingPageSidebar, to make it clearer that this is just to be used on the API landing pages?

One problem is that just about anything can be referred to as an API, from an entire spec of related functionality, to a single method. So maybe yours is better?

I’d be very happy to change DefaultAPISidebar so that it only includes the guides listed in groupdata, and we can pick and choose which ones we list, and the order in which they appear.

The only question I have left is — APIRef doesn’t list any of the guides that might exist that relate to that interface, or that method, etc. To find more out you have to go back to the API landing page. Are we happy with that? I guess that’s OK, and we are looking to simplify. To list specific guides per interface or whatever would make things too complex.

sheppy · May 13, 2019, 12:00pm

So… I think we need to have it both ways – homehow. We need to primarily list the ordered guides from the data, but we should find a way to also be sure every guide page in the hierarchy is also listed somewhere. Whether that’s in the sidebar, or perhaps in the content of the overview page doesn’t matter much to me. But some method to ensure that every guide shows up even when the writer forgets to add it to the database (which will happen in a world of so many contributors) is necessary, IMO.

I concur with the name SpecificationSidebar in principle although I’d suggest SpecSidebar instead because it’s shorter.

wbamberg · May 13, 2019, 6:12pm

I do think that APIRef should list guides, yes. I’ve so far avoided discussing the particular design of these sidebars, because I think it’ll be a morass, but I do think adding guide pages is an obviously useful thing to do.

In general I think the API sidebar experience is too fragmentary, but this issue intersects with the Great Mixin Debate, and in general how we should present the organization of APIs to users.

For example, suppose I’m interested in knowing about Fetch.

I start at https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API, where I get the GroupData sidebar.
I see Request, think it’s useful and visit https://developer.mozilla.org/en-US/docs/Web/API/Request. I now get the APIRef sidebar for Request. Looking at that page, I decide I want the json() method, so I click that.
I’m now at https://developer.mozilla.org/en-US/docs/Web/API/Body/json, and get the APIRef sidebar for Body.

That’s three closely related pages in the same quite small API, with three different sidebars - not just a bit different, but completely different. For me this makes navigation very difficult.

But yes, as a small step, I agree that including guides in APIRef seems like a good plan (also, now we have tests, we can do this with some degree of confidence).

Edited to add, yet again, sorry.

As well as guides, we could also add a link at the top of APIRef sidebars to the API overview page, which would help tie things together a bit.

So at the top of the sidebar on https://developer.mozilla.org/en-US/docs/Web/API/Body/json, we’d have a link to https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API.

wbamberg · May 13, 2019, 5:54pm

Well… I’m torn on this TBH. I do appreciate that it’s more convenient for guides to be listed automatically. But:

I think it’s important to list guides in a curated order. If we effectively make the group.guides property optional for subpages, then noone will use it.
It’s not like people are writing hundreds of guide pages every year. Here are all the subpages (level 1) of all the overview pages listed in GroupData (excluding the archived groups I want to remove): https://developer.mozilla.org/en-US/docs/User:wbamberg/all-guide-pages. There are 110 of them, and this is the product of several years’ worth of writing for all of Web/API.
This is similar to the way guides are added for static sidebars like AddonSidebar, LearnSidebar, JsSidebar, and it seems to work OK in these cases. It’s just part of the workflow: you write a guide, you update the sidebar.

exe-boss · May 14, 2019, 1:32am

{{APIRef}} supports this as long as you pass the correct group name argument.

wbamberg · May 14, 2019, 2:57am

Oh yes, sorry, I forgot about that.

sheppy · May 14, 2019, 4:02pm

Yes, definitely.

While I agree this is important, I think it’s also important that all guides get listed at all, and as things stand, with the sidebar having to be updated on GitHub, it’s far too easy to create a subpage that never gets added to the menus or sidebars. This is why I’ve always been in favor of some degree of automation, and why I think it’s important that we automatically add to the end of the page list any articles not included in the curated article list.

I would be more in favor of requiring every page to be manually added to the sidebar if editing content and metadata and the like all had a more integrated experience.

exe-boss · May 14, 2019, 10:46pm

That is already supported, as I explained in my reply:

wbamberg · May 14, 2019, 11:59pm

Web/API broken links

The Specifications section of Web/API uses the ListGroups macro to list all groups in GroupData, as links to the group’s API overview page. Many of these links are broken, which contributes to this page looking sad and unmaintained. Sometimes real users complain about this.

I’ve been through the broken links, excluding those that would be removed by my PR to clean up GroupData. The table below lists them all, together with the reason for the broken link.

API                             Reason for bad link
------------------------------------------------------------
CSSOM View                   -> no API overview page
Console API                  -> no API overview page
DOM Events                   -> no API overview page
Device Orientation Events    -> no API overview page
File API                     -> no API overview page
Geometry Interfaces          -> no API overview page
Image Capture API            -> needs overview property
Media Source Extensions      -> needs overview property
Resize Observer API          -> no API overview page
SVG                          -> no API overview page
Screen Orientation API       -> no API overview page
URL API                      -> no API overview page
Web Components               -> not under web/api
Web MIDI API                 -> no API overview page
WebVTT                       -> needs overview property
XDomain                      -> remove?

Nonexistent overview pages

So in most of these cases the problem is the obvious and not cheap-to-fix one: the overview page just doesn’t exist. We should file bugs to write overview pages for all these APIs.

Another thing we could do is this. At the moment, ListGroups will use the group’s overview property to build a link, if the overview property exists. But if the overview property does not exist, ListGroups will use the group’s name to build a link (https://github.com/mdn/kumascript/blob/master/macros/ListGroups.ejs#L45-L52). This means that an author can specify the group explicitly using overview or implicitly using the group name. I think we should stop doing this, and only use overview, and update GroupData to set overview for all groups that actually have overview pages.

This means ListGroups would stop showing broken links for nonexistent overview pages.

Also note that this ListGroups behavior is inconsistent with APIRef and DefaultAPISidebar, which only use the overview property to get an overview URL. So making this change would make all three macros consistent, and remove a bit of obscure magic from the ListGroups macro.

There are also a couple of nonexistent groups which are a bit different.

“CSSOM View”: this doesn’t have an overview page, but the overview page for CSSOM lists all the “CSSOM View” interfaces, treating them as a single thing. Perhaps we should fold “CSSOM View” into “CSSOM”.
“DOM Events”: this doesn’t have an overview page, but the overview page for DOM lists all the “DOM Events” interfaces, treating them as a single thing. In this case there’s even overlap in the interfaces the groups contain: Event and EventTarget appear in both “DOM” and “DOM Events”.

Other excuses

Apart from these, there are some easy fixes:

“Image Capture API”, “Media Source Extensions”, and “WebVTT” all have overview pages but the implicit name-derived links are incorrect, so this can be fixed just by giving them explicit overview properties.
“Web Components” already has an overview page but it’s not under Web/API, it’s at https://developer.mozilla.org/en-US/docs/Web/Web_Components. We should move it, right?
We’re surely never going to write an overview page for “XDomain”, so I suggest deleting this group (not deleting the interface page itself, just the group entry).

chrisdavidmills · May 15, 2019, 7:59am

@wbamberg there isn’t really anything I disagree with here. I think creating landing pages for these API is important, and something I would like to do early on as part of the maintenance project. Perhaps we could file an Epic and substories for “Fix API landing pages”, and do some estimates on how long it might take, then look at wherre we could fit it in, in Q3? I’m not saying you should do that necessarily; I’d be happy to help.

For web components, I ended up putting the landing page in a different place because I wasn’t so sure about it being an API in the same way as the others — it’s kind of a couple of related APIs, plus a couple of HTML elements, and some other bits.

We could argue that the Custom Elements API and the Shadow DOM API should have their own landing pages…but they make more sense being bundled together as web components, IMO.

But I don’t feel particularly strongly about this. I would be OK with moving it under Web/API.

wbamberg · May 15, 2019, 5:13pm

Thanks Chris, I have filed https://github.com/mdn/sprints/issues/1537 for adding the new pages. I didn’t create all the sub-issues yet though :).

Ah, that is interesting and a thing I hadn’t thought of. Then I think we should probably delete this group (since it’s not a “Web API” and GroupData is quite explicitly for Web APIs).

exe-boss · May 15, 2019, 10:52pm

I believe that the idea with “DOM Events” was that it would’ve been used back when events were still listed under “Web/Events”.

Also, I’ve now opened the PR to fold “CSSOM View” into “CSS Object Model”:

wbamberg · May 17, 2019, 10:03pm

@sheppy, how would we then avoid listing pages twice when they move? As far as I know this information isn’t available to macros.

Please, let’s try doing the simple thing now, that eliminates duplicates and gives us properly ordered guide pages, and if it becomes a problem to maintain, reconsider.

exe-boss · May 17, 2019, 10:46pm

Redirects don’t have tags, so when they’re moved (unless the guide is defined in GroupData.json), only the new location will be included.