Take the 2-minute tour ×

From reading the TeX source, it seems that the input_ln procedure is responsible for removing trailing spaces from lines upon input (before category codes are considered). A very short excerpt from the code there is

if buffer[last-1]<>" " then last_nonblank:=last;

The test appears to target specifically spaces, and not tab characters. However, doing some tests, I see that TeX also removes tabs (regardless of catcodes). For instance, running

echo -e "\\\\catcode9=12\\\\show\\t%" | tex

correctly shows the character <tab>, but omitting the % there is no output.

So... Which part of TeX removes trailing tabs?

share|improve this question
The real code for input_ln is in the change file, because it's system specific. –  egreg Sep 14 '12 at 20:22
add comment

2 Answers

up vote 12 down vote accepted

Here's how it goes in texlive (and presumably other web2c distros):

  1. The changefile tex.ch removes the Knuth definition of input_ln;
  2. The underscore gets lost in the conversion of web to c;
  3. The resulting C code #includes texmfmp.h which does

    #define inputln(stream, flag) input_line (stream) 
  4. The function input_line() gets defined in texmfmp.c where the trailing whitespace is cut thusly:

    /* Trim trailing whitespace.  */
    while (last > first && ISBLANK (buffer[last - 1]))
  5. ISBLANK() is defined in c-ctype.h as

    #define ISBLANK(c) (isascii (c) && isblank (c))

    where isblank() means (at least for usual locale settings) "is space or tab".

I'd say this behaviour could be a "bug", in that it deviates from the TeXbook: in chapter 8, page 46, Knuth writes:

TeX deletes any <space> characters (number 32) that occur at the right end of an input line. Then it inserts a <return> character (number 13) at the right end of the line, except that it places nothing additional at the end of a line that you inserted with 'I' during error recovery. Note that <return> is considered to be an actual character that is part of the line; you can obtain special effects by changing its catcode.

Elsewhere, we learn that plain tex sets the catcode for <tab> to 10 (space), and makes \<tab> the same as \<space> (and \<return>).

On the other hand, I can imagine the potential for confusion if tabs would not be skipped, here, so it makes some sense to do it.

share|improve this answer
Very good explanation, thanks! –  egreg Sep 18 '12 at 9:47
Very nice explanation. Your first point implies that Knuth's input_ln is different from the one in TeXlive. Does Knuth's remove tabs as well? -- EDIT: well, that comment of mine was silly: I had forgotten that my question specifically mentions that Knuth's input_ln only removes spaces. –  Bruno Le Floch Sep 18 '12 at 12:50
add comment

I think the get_next procedure (starting in Section 332 "Getting the next token") takes care of discarding tabs.

Section 343 "Input from external file ..." loads cur_char from buffer and sets cur_cmd to the catcode for cur_char; it then advances loc, the index into buffer, by 1. In Section 344 "Change state if necessary ...," TeX does nothing if none of the cases for the case statement match. Therefore, when state is skip_blanks, and cur_cmd is 10 (space), the othercases label will match and TeX will jump to the switch label at the beginning of Section 343. loc will be incremented and the tab (or space) will effectively be discarded.

Section 347 "Handle situations involving spaces, ..." puts TeX into the skip_blanks state, and also takes TeX out of skip_blanks when non-space characters are encountered.

share|improve this answer
I think what you describe is a different step: skip_blanks ignores any character which has category code 10. It turns out that tabs normally have this category code, but if you do \catcode9=12\relax then \show<tab>% and \show<tab>, you'll see first the character ^^I, then nothing, since \show is still looking for an argument. –  Bruno Le Floch Aug 15 '12 at 19:43
I have been looking through the sources and I cannot see where tabs get eaten! Is it possible the implementation of get in Web2c has some tab-eating behavior? Otherwise, I think you should send in a bug report :) –  Justin Bailey Aug 16 '12 at 19:57
@BrunoLeFloch From looking at the web2c sources, it looks like get is just implemented with getc (from C's standard library), which doesn't eat tabs. –  Justin Bailey Aug 17 '12 at 15:41
the behaviour is clearly by design, so no bug report there. This is very mysterious :(. –  Bruno Le Floch Aug 19 '12 at 0:39
add comment

Your Answer


