Htydjtk

I've tried for days to get my head around the expansion of code. Sadly, I don't get it.

I try to pass a command with arguments as an argument to sout.

documentclass{article}

usepackage{ulem,lipsum}

textwidth=3cm % just to force hyphenation/linebreaks to happen

usepackage[T1]{fontenc}



newcommand*{rom}[1]{romannumeral #1}



newcommandINTiintKATAkatNUMinum{% A bunch of commands get’s constructed by a lyx extension

Test Text for demonstration. In the original this command is constructed. %

}



newcommand{Kat}[4]{% Command to piece together the constructed commands.

 expandaftercsname INTrom{#1}intKAT#2katNUMrom{#3}num#4endcsname%

}



newcommandsoutt[1]{% sout with expansion

expandaftersoutexpandafter{#1}%

}



begin{document}

 sout{The sout makro of ulem is able to handle hyphenation and linebreaks in text entered directly but not as macro see next line!}

 soutt{INTiintKATAkatNUMinum} %works

 soutt{Kat{1}{A}{1}{}} %doesn't

end{document}

Any help is very much appreciated.

You should be aware of the fact that Kat requires more than one expansion step in order to deliver the stored text: after the first expansion step you get

expandaftercsname INTrom{1}intKATAkatNUMrom{1}numendcsname

This expandafter does nothing at all, because it tries to expand I.

You can force “almost full” expansion, which is what you need here, with the `romannumeral trick:

documentclass{article}

usepackage{ulem,lipsum}

usepackage[T1]{fontenc}



newcommand*{rom}[1]{romannumeral #1}



newcommandINTiintKATAkatNUMinum{%

  Test Text for demonstration. In the original this command is constructed. %

}



newcommand{Kat}[4]{% Command to piece together the constructed commands.

  csname INTrom{#1}intKAT#2katNUMrom{#3}num#4endcsname

}



newcommandsoutt[1]{% sout with expansion

  expandaftersoutexpandafter{romannumeral-`Q#1}%

}



begin{document}



parbox{2cm}{

 sout{The sout makro of ulem is able to handle hyphenation and linebreaks in text entered directly but not as macro see next line!}

 soutt{INTiintKATAkatNUMinum} %works

 soutt{Kat{1}{A}{1}{}} %doesn't

}



end{document}

And no, sout doesn't do hyphenation.

The romannumeral trick has been explained several times, but here's a brief illustration.

The primitive romannumeral wants a <number> after it and here we exploit a peculiar feature of TeX: after an explicit <number>, TeX looks for an <optional space> after it, expanding tokens during this lookup; it stops expansion upon finding either an explicit space token or an unexpandable token.

An “explicit <number>” can be expressed in several ways:

• a sequence of digits;


• a hexadecimal number, that is, a sequence of digits or characters among ABCDEF, if the first token is ";


• an octal number, that is, a sequence of digits among 01234567 if the first token is ';


• an alphabetical constant, if the first token is ` (a backquote).

In all cases, a minus sign should precede the radix notation "'`, if we want to specify a negative number.

In the first three cases, TeX performs expansion until finding something that doesn't qualify as an admissible digit.

The case that interests us is the last one. An alphabetic constant is a character token or a control sequence whose name consists of a single character (and need not be defined). So

`Q  `Q

are equivalent ways to ask for number 81 (the ASCII code of Q). The escaped notation is essential for characters with “strange” category code, such as

`% `^^@

(the latter is 0).

The fact that TeX expands tokens after the alphabetical constants is decisive for this application: the <number> is already fully known, and in our case it is -81, but TeX expands tokens nonetheless. After ending the search for the optional space, TeX will proceed to finalize the expansion of romannumeral-`Q, which is empty because the number is negative.

We cannot use romannumeral-81 directly, because if the following token after expansion happens to be a digit, it would be taken as part of the <number>.

A limitation of this method is that an initial space in the final expansion will be gobbled.

In the next release of TeX Live, all engines will have expanded (currently only available with LuaTeX); MiKTeX based engines should already have it. With expanded, the thing will be simpler:

newcommandsoutt[1]{% sout with expansion

  expandaftersoutexpandafter{expanded{#1}}

}

because expanded delivers the full expansion of its argument in a single step.

You can use soul, if you want hyphenation. The corresponding macro is st (or textst). Just for a change, I'll implement it in expl3, where the romannumeral is available as “f-expansion”.

documentclass{article}

usepackage[T1]{fontenc}

usepackage{soul}

usepackage{xparse}



ExplSyntaxOn



NewDocumentCommand{Kat}{mmmm}

 {

  use:c

   {

    INT int_to_roman:n {#1} intKAT #2 katNUM int_to_roman:n {#3} num #4

   }

 }

NewExpandableDocumentCommand{soutt}{m}

 {

  oberreiter_sout:f { #1 }

 }

cs_new_protected:Nn oberreiter_sout:n { textst{#1} }

cs_generate_variant:Nn oberreiter_sout:n { f }



ExplSyntaxOff



newcommandINTiintKATAkatNUMinum{%

  Test Text for demonstration. In the original this command is constructed. %

}



begin{document}



parbox{2cm}{

 st{The st macro of soul is able to handle hyphenation and linebreaks

   in text entered directly but not as macro see next line!}

 soutt{INTiintKATAkatNUMinum} %works

 soutt{Kat{1}{A}{1}{}} %doesn't

}



end{document}

Inspired by egreg's answer—where he exhibits the romannumeral-`Q⟨argument⟩-trick for keeping expansion with the leading token of the ⟨argument⟩ going until obtaining a leading token which is not expandable, whereby romannumeral-expansion might "eat" a leading space-token from the argument—I just wrote a routine HitArgsFirstTokenWithExpandafters which does "hit" the leading first token of its undelimited/curly-brace-nested argument with expandafter until obtaining a leading token which is not expandable.

Even if the argument was a single token, not nested in curly braces, the result will be delivered nested in curly braces.

The gist of the routine is:

A loop is initiated:

The routine checks whether the argument is empty or whether the argument's leading token is a space or an opening brace.

If this is the case, the routine is done as emptiness or leading braces or leading spaces imply that there is no expandable leading token.

If this is not the case, one can easily extract the leading token from the argument and check expandability of that token via expandafterifxnoexpand⟨token⟩⟨token⟩⟨not expandable⟩else ⟨expandable⟩fi and in case of expandability produce a hit via expandafter before initiating the next iteration.

Be aware that this routine does not expand the argument totally. Like the romannumeral-`Q⟨argument⟩-trick it aims only at the very first token of the argument (but unlike the romannumeral-`Q⟨argument⟩-trick it will not "eat" a leading space-token).

Thus breaking of lines will still be broken in case the argument contains macro tokens but not as leading tokens but somewhere behind the (fully expanded) leading tokens.

Be aware that such routines/tricks might cause problems, e.g., with unbalanced braces in case there are leading expandable token-sequences like
expandafter@gobblestring{!!!Attention!!! the closing brace will be unbalanced}.

Other approaches aiming at expanding the argument totally and already mentioned in egreg's answer are using expanded with LuaTeX-based engines and applying some variant of edef—I suggest protected@edef—with engines that don't have the expanded-primitive.

documentclass{article}

usepackage{ulem,lipsum}

%usepackage[T1]{fontenc}

usepackage{ifluatex}



newcommand*{rom}[1]{romannumeral #1}



newcommandINTiintKATAkatNUMinum{ A bunch of commands gets constructed by a lyx extension

  Test Text for demonstration. In the original this command is constructed. %

}



newcommand{Kat}[4]{% Command to piece together the constructed commands.

  expandaftercsname INTrom{#1}intKAT#2katNUMrom{#3}num#4endcsname

}%



% a redefinition of sout which lets you see what tokens sout gets as argument:

%defsout#1{deftest{#1}texttt{meaningtest}}%



newcommandsoutt[1]{% expandable sout with total expansion of first token of argument

  expandafterexpandafterexpandaftersoutHitArgsFirstTokenWithExpandafters{#1}%

}%



% You can use protected@edef for defining a temporary macro to expand

% to the total expansion of the argument before passing the expansion of

% that temporary macro to sout:



newcommandsouttb[1]{% non-expandable sout with total expansion of entire argument

  csname protected@edefendcsnamemytempa{#1}%

  expandaftersoutexpandafter{mytempa}%

}%



% In case of using LuaLaTeX you can use the expanded-primitive:



ifluatex

newcommandsouttc[1]{% expandable sout with total expansion of entire argument

  expandaftersoutexpandafter{expanded{#1}}%

}%

fi





makeatletter

%%-----------------------------------------------------------------------------

%% Paraphernalia ;-) :

%%.............................................................................

newcommandUD@firstoftwo[2]{#1}%

newcommandUD@secondoftwo[2]{#2}%

newcommandUD@exchange[2]{#2#1}%

%%-----------------------------------------------------------------------------

%% Check whether argument is empty:

%%.............................................................................

%% UD@CheckWhetherNull{<Argument which is to be checked>}%

%%                     {<Tokens to be delivered in case that argument

%%                       which is to be checked is empty>}%

%%                     {<Tokens to be delivered in case that argument

%%                       which is to be checked is not empty>}%

%% The gist of this macro comes from Robert R. Schneck's ifempty-macro:

%% <https://groups.google.com/forum/#!original/comp.text.tex/kuOEIQIrElc/lUg37FmhA74J>

%%.............................................................................

newcommandUD@CheckWhetherNull[1]{%

  romannumeral0expandafterUD@secondoftwostring{expandafter

  UD@secondoftwoexpandafter{expandafter{string#1}expandafter

  UD@secondoftwostring}expandafterUD@firstoftwoexpandafter{expandafter

  UD@secondoftwostring}expandafterexpandafterUD@firstoftwo{ }{}%

  UD@secondoftwo}{expandafterexpandafterUD@firstoftwo{ }{}UD@firstoftwo}%

}%

%%-----------------------------------------------------------------------------

%% Check whether argument's first token is a catcode-1-character

%%.............................................................................

%% UD@CheckWhetherBrace{<Argument which is to be checked>}%

%%                      {<Tokens to be delivered in case that argument

%%                        which is to be checked has leading

%%                        catcode-1-token>}%

%%                      {<Tokens to be delivered in case that argument

%%                        which is to be checked has no leading

%%                        catcode-1-token>}%

newcommandUD@CheckWhetherBrace[1]{%

  romannumeral0expandafterUD@secondoftwoexpandafter{expandafter{%

  string#1.}expandafterUD@firstoftwoexpandafter{expandafter

  UD@secondoftwostring}expandafterexpandafterUD@firstoftwo{ }{}%

  UD@firstoftwo}{expandafterexpandafterUD@firstoftwo{ }{}UD@secondoftwo}%

}%

%%-----------------------------------------------------------------------------

%% Check whether brace-balanced argument starts with a space-token

%%.............................................................................

%% UD@CheckWhetherLeadingSpace{<Argument which is to be checked>}%

%%                             {<Tokens to be delivered in case <argument

%%                               which is to be checked>'s 1st token is a

%%                               space-token>}%

%%                             {<Tokens to be delivered in case <argument

%%                               which is to be checked>'s 1st token is not

%%                               a space-token>}%

newcommandUD@CheckWhetherLeadingSpace[1]{%

  romannumeral0UD@CheckWhetherNull{#1}%

  {expandafterexpandafterUD@firstoftwo{ }{}UD@secondoftwo}%

  {expandafterUD@secondoftwostring{UD@CheckWhetherLeadingSpaceB.#1 }{}}%

}%

newcommandUD@CheckWhetherLeadingSpaceB{}%

longdefUD@CheckWhetherLeadingSpaceB#1 {%

  expandafterUD@CheckWhetherNullexpandafter{UD@secondoftwo#1{}}%

  {UD@exchange{UD@firstoftwo}}{UD@exchange{UD@secondoftwo}}%

  {UD@exchange{ }{expandafterexpandafterexpandafterexpandafter

   expandafterexpandafterexpandafter}expandafterexpandafter

   expandafter}expandafterUD@secondoftwoexpandafter{string}%

}%

%%-----------------------------------------------------------------------------

%% Extract first inner undelimited argument:

%%.............................................................................

%%   UD@ExtractFirstArg{ABCDE} yields  {A}

%%

%%   UD@ExtractFirstArg{{AB}CDE} yields  {AB}

%%.............................................................................

newcommandUD@RemoveTillUD@SelDOm{}%

longdefUD@RemoveTillUD@SelDOm#1#2UD@SelDOm{{#1}}%

newcommandUD@ExtractFirstArg[1]{%

  romannumeral0%

  UD@ExtractFirstArgLoop{#1UD@SelDOm}%

}%

newcommandUD@ExtractFirstArgLoop[1]{%

  expandafterUD@CheckWhetherNullexpandafter{UD@firstoftwo{}#1}%

  { #1}%

  {expandafterUD@ExtractFirstArgLoopexpandafter{UD@RemoveTillUD@SelDOm#1}}%

}%

%%-----------------------------------------------------------------------------

%% Check whether argument's first token is expandable

%%.............................................................................

%% UD@CheckWhetherFirstTokenExpandable{<Argument which is to be checked>}%

%%                      {<Tokens to be delivered in case that 

%%                         <argument which is to be checked> has a first

%%                         token which is expandable>}%

%%                      {<Tokens to be delivered in case that

%%                         <argument which is to be checked> does not have

%%                         a first token which is expandable>}%

%%

newcommandUD@CheckWhetherFirstTokenExpandable[1]{%

  romannumeral0%

  UD@CheckWhetherNull{#1}{UD@exchange{ }{expandafter}UD@secondoftwo}{%

    UD@CheckWhetherBrace{#1}{UD@exchange{ }{expandafter}UD@secondoftwo}{%

      UD@CheckWhetherLeadingSpace{#1}{UD@exchange{ }{expandafter}UD@secondoftwo}{%

        expandafterexpandafterexpandafterUD@@CheckWhetherFirstTokenExpandable

        UD@ExtractFirstArg{#1}%

      }%

    }%

  }%

}%

newcommandUD@@CheckWhetherFirstTokenExpandable[1]{%

  expandafterifxnoexpand#1#1%

  expandafterUD@firstoftwoelseexpandafterUD@secondoftwofi

  {UD@exchange{ }{expandafter}UD@secondoftwo}%

  {UD@exchange{ }{expandafter}UD@firstoftwo}%

}%

%%-----------------------------------------------------------------------------

%% Hit argument's first token with expandafter until argument's first token is

%% not an expandable token. Then deliver that nested in braces.

%%.............................................................................

newcommandHitArgsFirstTokenWithExpandafters[1]{%

  romannumeral

  UD@CheckWhetherFirstTokenExpandable{#1}{%

     expandafterUD@firstoftwoexpandafter{expandafter}%

     romannumeral0UD@exchange{ }{expandafterexpandafterexpandafter}%

     expandafterHitArgsFirstTokenWithExpandaftersexpandafter{#1}%

  }{0 {#1}}%

}%

%%

%% With all the macros above, the result is delivered after two expansion-steps 

%% / after two expandafter-chains due to  romannumeral-expansion.

makeatother



parindent=0ex

parskip=smallskipamount



begin{document}

vspace*{-4.5cm}%

enlargethispage{8cm}%

pagestyle{empty}%

The sout makro of ulem does suppress hyphenation in any case.



hrulefillnull



begin{verbatim}

parbox{4.5cm}{%

  sout{The sout makro of ulem is able to handle linebreaks in text

        entered directly but not as macro!}

}%

end{verbatim}%

parbox{5cm}{%

  sout{The sout makro of ulem is able to handle linebreaks in text

        entered directly but not as macro!}

}%



hrulefillnull



Linebreaking is not disturbed with the following:



verb|parbox{5cm}{soutt{INTiintKATAkatNUMinum}}|



parbox{5cm}{soutt{INTiintKATAkatNUMinum}}%



dotfillnull



verb|parbox{5cm}{soutt{Kat{1}{A}{1}{}}}|



parbox{5cm}{soutt{Kat{1}{A}{1}{}}}



hrulefillnull



Linebreaking is disturbed with the following:



verb|parbox{5cm}{soutt{Breaking of lines is broken: INTiintKATAkatNUMinum}}|



parbox{5cm}{soutt{Breaking of lines is broken: INTiintKATAkatNUMinum}}



dotfillnull



verb|parbox{5cm}{soutt{Breaking of lines is broken: Kat{1}{A}{1}{}}}|



parbox{5cm}{soutt{Breaking of lines is broken: Kat{1}{A}{1}{}}}



hrulefillnull



Linebreaking is not disturbed with the following:



verb|parbox{5cm}{souttb{Breaking of lines is not broken: INTiintKATAkatNUMinum}}|



parbox{5cm}{souttb{Breaking of lines is not broken: INTiintKATAkatNUMinum}}



dotfillnull



verb|parbox{5cm}{souttb{Breaking of lines is not broken: Kat{1}{A}{1}{}}}|



parbox{5cm}{souttb{Breaking of lines is not broken: Kat{1}{A}{1}{}}}



ifluatexdotfillnull



verb|parbox{5cm}{souttc{Breaking of lines is not broken: INTiintKATAkatNUMinum}}|



parbox{5cm}{souttc{Breaking of lines is not broken: INTiintKATAkatNUMinum}}



dotfillnull



verb|parbox{5cm}{souttc{Breaking of lines is not broken: Kat{1}{A}{1}{}}}|



parbox{5cm}{souttc{Breaking of lines is not broken: Kat{1}{A}{1}{}}}

fi



end{document}