Wikisource:Labeled section transclusion

Labeled Section Transclusion is a function that enables marked sections of text to be transcluded. This function was originally developed with the needs of Wikisource in mind.

For up-to-date information on this Mediawiki extension, please see:
Extension:Labeled Section Transclusion

This page is inactive and has been retained for historical interest.
You can help by documenting its historical context and relevance. If you want to revive discussion regarding the subject, you are welcome to start a new discussion at the Scriptorium.

Introduction

Here is a suggestion for enabling sections of texts at Wikisource to be transcluded. The current statement of the concept as code (in XML-style syntax) is thanks to User:Eloquence, who strongly suggests that the function be termed "Labeled section transclusion" (as we have called it on this page).

The following sections briefly describe how labeled section transclusion would work, and its rationale.

How it works

A. Mark off a section of text

A section of text (an individual chapter, subsection of a chapter, a numbered paragraph in a classical text, an individual verse) might be marked off something like this:

<begin section="Paragraph 1"/> ...text of the section...<end section="Paragraph 1"/>

The slashes inside the tags are XML-style, which is similar to <br/>, <references/>, etc.

NB: Sections may overlap (different sectioning mechanisms may be used within the same text), so a syntax where the end tag is ambiguous is not acceptable.

B. Call up the section of text

When calling up that section, it would be done like this:

{{Text##Paragraph 1}}

The double-## syntax is suggested because a single "#" has the problem of being normally used for regular wiki sections, so users might intuitively try transcluding sections and be confused when it doesn't work. That is why "##" is suggested as an alternative. Other non-title characters would work, too.

Previous coding suggestion

The first clear statement of this concept as code used ParserFunctions, and is thanks to User:Pathoschild.

Mark off a section of text like this:

{{#section:name|start}} ...text... {{#section:name|end}}

Call up that section of text like this:

{{pagename#section:name}} or {{pagename#name}}

Though the preceeding coding syntax (XML-style) seems to be more appropriate according to the Mediawiki developers, this statement by User:Pathoschild was the first success at stating the idea clearly, and subsequent developments are thanks to his help!

Rationale

Those who study classical texts quickly learn that they have what we might call today a "hypertext" nature. This means that generations of scholars over many centuries refer to and cite each others' works, all within the same corpus. This results in a kind of literature that consists of primary texts (often in multiple versions), multiple commentaries on them, multiple super-commentaries, and even super-super-commentaries.

This scenario accurately describes the literature of Aristotelian science, rabbinic literature, classic church literature, Islamic literature, and is probably true for other cultures as well.

Also, the clear division of books into large subsections called "chapters," which is typical of the modern books at Wikisource, is very different from the numerous complementary ways that ancient, classical, and medieval texts were divided, commented on, and cited.

At present, there is one very specific feature missing in Mediawiki that would be of enormous assistance for pre-modern literature. Such a feature would allow the creation of different editions of the same corpus with no duplication of text:

Editions that present parallel versions (such as parallel translations) of individual passages and compare them;
Editions which present collections of commentaries on individual passages;
Editions which allow a commentary to be presented both in running form and/or in individual units together with the primary text that it comments on.

Currently, the only way to build the above editions is by putting each small unit of text on its own page, and then transcluding masses of such smaller units every time you call up the commentaries on one verse, or present one chapter based on transcluding individual verses. Otherwise the texts have to be duplicated a great many times, something which is unhealthy for a number of reasons.

Small Wikisource text projects that have tried to implement the presentation of such texts under present circumstances find that they need to create thousands of pages, each with just a small unit of text, just in order to present variations of what is not a large quantity of text at all. These clutter up the template namespace or the main namespace, and call for a namespace of their own (perhaps to be called the "Passage:" namespace, for individual passages of a text). All of this just to cover a small amount of text.

Creating slightly larger works based upon such texts would eventually require tens or hundreds of thousands of individual pages, and hundreds of thousands of cumbersome manual actions on the wiki.

Having what is intuitively the opposite function (i.e. putting the entire version of a book on one page with invisible section markers, and then transcluding portions of it in relevant places) would be much more feasible and far less messy.

Currently, the Bible and its commentaries provide an excellent example of the problem (but they are by no means the only example). At Hebrew Wikisource, for instance, just to present the very short book of Jonah properly with its commentaries required the creation of literally hundreds of transcluded text-units (the work was done by User:Nevuchadnezzar). The process currently works, but not in a reasonable way.

The enabling of labeled section transclusion would be of great assistance to Wikisource for Bible and for many other classical texts from a range of cultures.

Mediawiki extension feature

This tool could be an enormously useful: Any and all of the texts-units at Wikisource that should be presented in a variety of complementary forms, whether in terms of presenting parallel translations or collections of commentaries on individual units. Such a feature would remove the need to duplicate those units of text.

Bug report

This proposal for a Mediawiki extension may be found at:

Labeled Section Transclusion (Bug 5881)

Unfortunately, the Bugzilla proposal was only clarified towards the end of a very lengthy discussion. This wiki page reflects the current status of the proposal.

Please contribute to the discussion both on this page and at the bug, and add your vote to the bug.

Mediawiki extension

This proposal is listed as an extension feature to the Mediawiki software:

Labeled Section Transclusion

That page contains the most up-to-date description of the tool, which now includes all of the flexibility and features requested at this current Wikisource discussion, and even more.

Developer feedback

Developer feedback has been on two levels: content and coding.

Most reservations have been in terms of content, because this concept has less relevance to Wikipedia-style articles: Why would anyone want to mark of part of a Wikipedia article to be transcluded? However, even those who had such reservations had no problem with the idea that the function could be appropriate to Wikisource.
The above XML-style syntax is the latest suggestion for how this could be appropriately coded.

According to User:Eloquence, the coding for this function "should be manageable," though it would have to deal with a few technical issues that might be tricky:

Parsing the sections without choking: There may be typos, missing end markers, dupe sections, etc.
Indicating where a section is being used (refined Special:Whatlinkshere, needed for fixing broken transclusions).
Making sure all the proper table and cache purge operations are done whenever a section is added/modified/removed.

Discussion of technical issues continues at the talk page.

Wikisource opinion

This opinion poll is regarding the desirability of having labeled section transclusion enabled as an available tool for Wikisource. If made available in the future, it will be used for those texts in which the community deems it appropriate (and not for others). Having the tool available does not require its use in any or all cases.

This poll does not deal with the technical feasibility of implementing the function in the software (if it proves too difficult on a software level, it will obviously not be implemented anyway).

Note: If you support this, please vote for the bug as well.

Support:

Support Dovi 19:22, 31 May 2006 (UTC) (as author of the proposal)[reply]
Support.—Zhaladshar ^(Talk) 19:31, 31 May 2006 (UTC)[reply]
Support. --BirgitteSB 22:28, 31 May 2006 (UTC)[reply]
Support -- Danny 03:31, 1 June 2006 (UTC)[reply]
Support -- Banjee ca 02:15, 2 June 2006 (UTC)[reply]
Support -- Something like this would be extremely useful. Jude (talk,contribs,email) 23:21, 4 June 2006 (UTC)[reply]
Support --Kernigh 21:34, 8 June 2006 (UTC)[reply]
Support --HappyDog 17:28, 10 September 2006 (UTC)[reply]
Support, since I went through the trouble to implement it - Steve Sanbeg 16:06, 3 November 2006 (UTC)[reply]

Objections:

Object I hope it doesn't seem too presumptuous to be contrary to consensus, when I only got involved with wikisource during the last converence session. To me, this seems like feature creep. This would only be used in wikisource, and doesn't seem that difficult with the current functionality in the software. I don't think it's a good idea to add new syntax and software features for simple things.Sanbeg 15:51, 5 August 2006 (UTC)[reply]
- Note: If this is implemented using an extension then this objection becomes irrelevant. --HappyDog 17:28, 10 September 2006 (UTC)[reply]
  - Interesting thought; although my understanding of extensions (which admittedly is still evolving) is that they can't tell if they are called from a template, much less get template args (or args from a new syntax). I'd think the markers would be similar to the onlyinclude implementation. If we really wnat the special calling syntax, that would be complicate things, but I haven't seen much specification for what that would do. Of course, whole my objection was more due to limited specifications about why this differs from what's already there than being opposed to a feature if necessary. -Sanbeg 17:36, 11 September 2006 (UTC)[reply]
  - My bad, I've now figured out extensions well enough to see that this can be done with parser functions after all. So I guess my objection is irrelevant. -Steve Sanbeg 16:06, 3 November 2006 (UTC)[reply]

Important discussion of technical alternatives in the wake of the above comment may be found at the talk page. Summaries of several alternatives from that discussion follow in the next section.

Alternative solutions

See the talk page for extensive discussion of the alternative solutions summarized in the following subsections.

Transclusion of multiple small elements

Put each individual small passage of text on its own separate page (ideally there should be a separate namespace for these small sections) and transclude them individually in whatever multiple contexts they should appear in.

Implemented at Hebrew Wikisource. Check out the individual verses of the Book of Jonah with commentaries, whose index is here. This small book transcluded hundreds of small elements, and there are other texts like it at he:. (The small elements are currently in the "template" namespace, but are planned to be moved to the new "passage" namespace, hopefully via bot.)

Disadvantage: This method is extremely labor-intensive, and also involves the creation of huge numbers of separate pages.

Text in document based on "onlyinclude", with #switch in helper template

Proposed by User:Sanbeg and implemented for examples at English Wikisource.

Text from the source document is marked in numbered sections, which can use partial substitution or #if to allow the section to be suppressed; onlyinclude is used to separate the sections from parts of the document that should not be transcluded.

A helper template contains the logic to suppress the unwanted sections during transclusion.

Bible/Philemon is a list of 25 pages that were generated with this method; each transcludes a section from 4 versions of the same text.

Text within template based on "if"

Implemented at Hebrew Wikisource, this solution is similar to that of User:Sanbeg in that the text is put into a template, but it utilizes the parser function "if" rather than "onlyinclude".

This has been implemented extensively at Hebrew Wikisource by he:User:Sgvb. For an example, see the wiki-code for this text. The page itself appears blank because it is in a template; sections of it are called up on other pages.

Hidden sections

When a page is transcluded, selectively hide desired parts of it by using parameters. Implemented at French Wikisource by User:ThomasV.

Example: fr:« C'était alors, quand, les chaleurs passées, » The transcluded page uses the template "NouveauChapitre" as a delimiter.

The template "Page" is called with two parameters, "from" and "to".

Example: fr:Essais_-_Livre_I_-_Chapitre_6
User's manuel: fr:Aide:Affichage_par_pages (French)
Also see: Wikisource:Subdomains coordination

Extension

This functionality is now available as an extension, at http://svn.wikimedia.org/viewvc/mediawiki/trunk/extensions/LabeledSectionTransclusion/, see mw:Labeled Section Transclusion for more information. This should be available to any project that needs it by opening a wikimedia bug for the project. -Steve Sanbeg 16:45, 17 November 2006 (UTC)[reply]

Great work! So shall we open a new bug to have it requested for Wikisource? What is the best way of going about this? Dovi 21:08, 18 November 2006 (UTC)[reply]

Since it's in SVN now, I think our next step is just to have a dev enable it on this project.—Zhaladshar ^(Talk) 22:07, 18 November 2006 (UTC)[reply]

My question is: Should be just open a bug? (Perhaps it would be best for User:Sanbeg to do that, since he knows exactly what we are asking for and how to implement it.) But even when we open the bug, what will the developers ask for: For each language to decide individually? Or is it enough that we have clear support here in English (plus French and Hebrew have also been searching for something like this)? I suggest that it would be best to state in the bug that this is an enhancement that does no harm, has an affinity to Wikisource, and should be implemented across Wikisource languages unless a language states a particular objection to it (which is highly unlikely). Dovi 19:42, 19 November 2006 (UTC)[reply]

I've opened the bug for en @ bugzilla:7995. I don't know if we need to demonstrate community or bureaucrat support for that to go, or how we'd put it on multiple languages. I think it's something that the devs & bcats need to agree on. Maybe the individual projects should add a note to this request, or file a similar one. -Steve Sanbeg 18:48, 20 November 2006 (UTC)[reply]

I've added an experimental #lsth function to transclude visual headings, we'll see how useful that is. Also, I noticed that most of the basic parser functions also have Hebrew aliases. Do we need something like that if we want to install that on he:? I.e. the #if function below. -Steve Sanbeg 19:45, 8 December 2006 (UTC)[reply]

            $magicWords['if']      = array( 0, 'תנאי',        'if' );

I think aliases is a good idea. Cryptic acronyms like lst, lstx or lsth will not make much sense in other languages than English (and not even in English). It is much better to have real words as names of functions. As examples of bad naming you only have to look at some of the templates in Wikipedia, you can't tell what these templates do without looking at the template pages: template:ult, template:tl, template:db (has nothing to to do with databases), ... /82.212.68.183 00:33, 9 December 2006 (UTC)[reply]

I'm not sure yet how it will localize in general. Unfortunately, labeledsectiontransclusion is just too long to be practical. Maybe we'll come up with another alternative that's more intuitive -Steve Sanbeg 02:03, 10 December 2006 (UTC)[reply]

My hunch is that even though Hebrew has such aliases (implemented by Rotem Liss), the "originals" still work anyways. The best person to ask is Rotem, who is available at the various developer forums, plus I will also ask him at he:. Dovi 20:33, 9 December 2006 (UTC)[reply]

I don't doubt they'll work, it just seems like an interesting coincidence that Hebrew seems to have the most localisation, and also wanted this extension. So it may be useful for someone to come up with Hebrew names for these as well. -Steve Sanbeg 01:43, 10 December 2006 (UTC)[reply]