PCGen Wiki - User contributions [en]

Setting up the new Formula System

2018-11-13T02:17:31Z

Tom Parker: /* getOther */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

The getOther function returns the value of a remote local variable.

The format of the getOtherfunction is:
getOther(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables and CAN have MODIFY/MODIFYOTHER tokens.

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
CHANNEL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)|PRIORITY=1000

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

===Scope Change===

Effective in about April 2018, the scopes changed. What was originally just EQUIPMENT became PC.EQUIPMENT. This was done in preparation for things that are not the PC (think familiars, among other things).

Setting up the new Formula System

2018-11-13T02:17:15Z

Tom Parker: /* getOther */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getOther(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables and CAN have MODIFY/MODIFYOTHER tokens.

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
CHANNEL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)|PRIORITY=1000

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

===Scope Change===

Effective in about April 2018, the scopes changed. What was originally just EQUIPMENT became PC.EQUIPMENT. This was done in preparation for things that are not the PC (think familiars, among other things).

Setting up the new Formula System

2018-11-13T01:54:46Z

Tom Parker: /* DYNAMIC (PCC File) */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables and CAN have MODIFY/MODIFYOTHER tokens.

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
CHANNEL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)|PRIORITY=1000

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

===Scope Change===

Effective in about April 2018, the scopes changed. What was originally just EQUIPMENT became PC.EQUIPMENT. This was done in preparation for things that are not the PC (think familiars, among other things).

Setting up the new Formula System

2018-11-13T01:45:51Z

Tom Parker: /* Material Changes */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
CHANNEL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)|PRIORITY=1000

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

===Scope Change===

Effective in about April 2018, the scopes changed. What was originally just EQUIPMENT became PC.EQUIPMENT. This was done in preparation for things that are not the PC (think familiars, among other things).

Setting up the new Formula System

2018-11-13T01:44:34Z

Tom Parker: /* An Example */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
CHANNEL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)|PRIORITY=1000

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:43:19Z

Tom Parker: /* CHANNEL (VARIABLE File) */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
CHANNEL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:42:11Z

Tom Parker: /* To be completed */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==Channels==

The internal processing system in general needs a method of communicating with the user interface. Traditionally we have done this through hard-coded systems, but with increased flexibility of the formula system, it makes less and less sense to have hard-coded capabilities. We also have scenarios such as Gender (and a few cursed objects) where an object needs to override user input (at least temporarily).

In order to do this, we are enabling items in the UI to communicate directly to the formula system. This will allow one to alter the other. There are two ways this is done:
* Channels
* Input function

Channels are the actual storage location (think of it as a special type of variable) that holds the information that is communicated between the core and the user interface. If either the core or the UI changes the value, it is changed. Channels can be created by the CHANNEL: token in the variable file.

As special variables, Channel names do NOT conflict with variable names, but they also have conflict rules that are identical to the variable naming rules as shown elsewhere on this page.

Note that some CodeControls ask for a Channel. This allows the data team to override the internal (and non-accessible) channel with one the data team can then use.

The new formula system can read a channel by using the input(x) function.

===CHANNEL (VARIABLE File)===

* CHANNEL defines a channel, usable from anywhere within the UI

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the Channel
* X is the format name
* Y is the channel name

Example:
CHANNEL:PC.STAT|NUMBER=StatInput

A CHANNEL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===input Function===

The input function returns the value of a given channel.

The format of the get function is:
input(x)

* x is a Channel name.

Note: You may be caught off guard here by the simplicity of this method [you aren't providing a scope]. If the Channel is defined locally to an object, such as the StatInput example above, then input("StatInput") will only work if called from the stat. So in equipment, calling input("StatInput") would generate an error, as there isn't a "StatInput" Channel in PC.EQUIPMENT. However, there are ways around that, such as:
getOther("PC.STAT","Dex",input("StatInput"))
...which would grab the "StatInput" channel from the Stat with the abbreviation "Dex".

===An Example===

In practice, Gender might be controlled with the following:

In the DataControl file:
DYNAMICSCOPE:GENDER
In a DYNAMIC File:
GENDER:Male <> MODIFY:Reversed|SET|Female
GENDER:Female <> MODIFY:Reversed|SET|Male
In the Variable file:
CHANNEL:PC|GENDER=InputGender
GLOBAL:GENDER=ActualGender
LOCAL:GENDER|GENDER=Inverted
Then in the global Modifier file:
MODIFY:ActualGender|SET|input("InputGender")
A cursed object would have:
MODIFY:ActualGender|SET|getFact("GENDER",value(),Reversed)

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:18:20Z

Tom Parker: /* Built in Functions */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

====key====

This function returns the key of an object (must be a CDOMObject)

The format of the get function is:
key(x)

* x is an Object (like a skill)

Example: key(myFavoredClass)
[This assumes myFavoredClass is a legal variable name that probably contains a CLASS]

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==To be completed==
* Channels

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:15:44Z

Tom Parker:

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

====get====

This function returns an actual object of the given type (such as retrieving a SKILL or ALIGNMENT). This is useful for other functions where this object is used.

The format of the get function is:
get(x,y)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is the key of a CDOMObject (like a Skill) (also in quotes)

Example: get("SKILL","Hide")

====getFact====

This function returns the value of a FACT.

The format of the getFact function is:
getFact(x,y,z)

* x is an Object type (like "SKILL") [technically this is the FORMAT, but not all are enabled as formats yet]
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the FACT to be returned

Example: getFact("SKILL","Hide","IsExclusive")

====getOther====

This function returns the value of a remote local variable.

The format of the getFact function is:
getFact(x,y,z)

* x is the remote scope from which an object is being retrieved (e.g. "PC.SKILL")
* y is a CDOMObject OR the key of a CDOMObject (like a Skill) (also in quotes)
* z is the name of the variable to be returned... note this is NOT in quotes

Example: getOther("PC.SKILL","Hide",IsExclusive)

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==To be completed==
* Channels

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:07:24Z

Tom Parker: /* Understanding SubScopes in relation to MODIFYOTHER */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

Additional functions specifically related to PCGen are available:
* get: This function takes two arguments. The first is an Object type (like "SKILL"), the second is the key of a CDOMObject (like a Skill) (also in quotes)
**Example: get("SKILL","Hide")
* getFact: This function returns the value of a FACT. The first two arguments are much like the two arguments of "get", the third argument is the name of the FACT to be returned

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:PC.EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==To be completed==
* getOther
* Channels

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:06:14Z

Tom Parker: /* No Overlapping */

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an PC.EQUIPMENT variable, then "CritMult" can't be an PC.EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local PC.STAT variable and a local PC.CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

Additional functions specifically related to PCGen are available:
* get: This function takes two arguments. The first is an Object type (like "SKILL"), the second is the key of a CDOMObject (like a Skill) (also in quotes)
**Example: get("SKILL","Hide")
* getFact: This function returns the value of a FACT. The first two arguments are much like the two arguments of "get", the third argument is the name of the FACT to be returned

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==To be completed==
* getOther
* Channels

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Setting up the new Formula System

2018-11-13T01:04:49Z

Tom Parker:

{| align="right"
| __TOC__
|}

=The basics=

The new formula system is designed to do a few things relative to data:
* Validate formulas at LST load instead of runtime
* Dramatically increase flexibility
* It will replace many tokens, including all the BONUS tokens
* It allows a sensible location for global modifications (things generally related to rule book type modifications that impact all objects of a given type)

==Key concepts==

To use the new system a few key things need to be remembered:
* It is possible to have local variables
* All variables must be defined prior to use
* Variables are not only numbers, but can be Strings, Dice, etc.
** We call this the variable FORMAT

You will want to keep track of what these key terms mean as you learn the new formula system:
* FORMAT: The type of variable, e.g. NUMBER, BOOLEAN, ORDEREDPAIR.
* SCOPE: The part of the data in which a (local) variable is available
* MODIFIER: An argument to MODIFY* tokens, specifically the "command" of what the modification represents
* OPERATOR: A mathematical operator as you would see in a formula ("+", "-", etc.)
* GROUPING: A name for a set of objects in PCGen
* ASSOCIATION: An additional set of information attached to a MODIFY* token. This is a key-value pair of information, e.g. PRIORITY=100

=Creating and Using Variables=

==Required Setup==

Every format that needs to be used requires a default value. This value is shared throughout an entire system (it CANNOT be redefined to a different value). This is placed into a file referred to in a PCC file from the DATACONTROL: token.

===DATACONTROL (PCC File)===

DATACONTROL: is a new Campaign file token (for PCC files). It is treated "recursively" like other tokens, e.g. TEMPLATE.

DATACONTROL:x

* x is the file to be loaded as a DATACONTROL file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

===DEFAULTVARIABLEVALUE (DATACONTROL LST)===

The Format is:
DEFAULTVARIABLEVALUE:X|Y
* X is the format name
* Y is the default value

Example:
DEFAULTVARIABLEVALUE:NUMBER|0

It is a best practice to always define a DEFAULTVARIABLEVALUE for every FORMAT regardless of whether the data uses that FORMAT. The internal parts of PCGen may use it, so a default value may be required.

Note: it IS safe to "redefine" a DEFAULTVARIABLEVALUE to the same value. So if multiple datasets are loaded and they all had DATACONTROL: and all defined the default value for NUMBER to be 0, they will all happily load as the system is capable of recognizing the defaults are the same

==How to define a variable==

===VARIABLE (PCC File)===

To define a variable, you need to add an LST file that is referred to in a PCC file from the VARIABLE: token. It is treated "recursively" like other tokens, e.g. TEMPLATE.

VARIABLE:x

* x is the file to be loaded as a VARIABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

There are 2 basic tokens for the VARIABLE file: GLOBAL and LOCAL.

===GLOBAL (VARIABLE File)===

* GLOBAL defines a global variable, usable from anywhere within the PlayerCharacter.

The format is:
GLOBAL:X=Y
* X is the format name
* Y is the variable name

Example:
GLOBAL:ORDEREDPAIR=Face

A GLOBAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===LOCAL (VARIABLE File)===

* LOCAL defines a local variable, usable only within the object on which the variable exists (or its children)
** The place where a local variable is usable is called the SCOPE. This could include a STAT (such as the stat's SCORE or STATMOD) or EQUIPMENT. The scopes possible in PCGen are hard-coded (see below for more info)

The format is:
LOCAL:W|X=Y
* W is the SCOPE of the variable
* X is the format name
* Y is the variable name

Example:
LOCAL:PC.STAT|NUMBER=Score

A LOCAL token can take additional token on the line called EXPLANATION, which it is advisable to use to communicate to other data monkeys (and it is likely any future editor will also draw on this information).

===EXPLANATION (VARIABLE File)===

EXPLANATION: is a free-text token (no parsing is really done) that is designed to describe why a variable exists

Format:
EXPLANATION:x

* x is the text of the Explanation
* Limitations: By Design: Must not appear as the first token on a line
* Behavior: Overwrites (as if someone would use it twice??)

===Defaults in the VARIABLE file===

A few notes about the VARIABLE files:
If you see an item without a leading token, e.g.:
ORDEREDPAIR=Face
...then the system is using the default (GLOBAL). For now, the data standard is to avoid using this shortcut (as far as I know)

If you see an item without a leading format ("X" in both GLOBAL and LOCAL above), e.g.:
LOCAL:STAT|Score
...then the format is a NUMBER. For now, the data standard is to avoid using this shortcut (as far as I know)

Note both defaults can be combined, so a variable name with no other information on that line is a Global NUMBER

===No Overlapping===

In most programming languages, it is legal to have local variable share the same name as the global variable and thus "override" it in that context (although it is considered less than ideal, and in some languages it will now produce a warning). It is not legal in PCGen.

If a Global variable called "Hands" exists, then NO local variable IN ANY SCOPE can be "Hands".

Similar restrictions exist for scopes and SubScopes. If "CritMult" is an EQUIPMENT variable, then "CritMult" can't be an EQUIPMENT.PART variable.

Non-overlapping (when the Scopes do not interact) is legal. "Score" can be a local STAT variable and a local CHECK variable, for example.

For clarity:
* This occurs regardless of the FORMAT of the variables - tokens will infer the format from the variable name, so you can't have overlapping variables even of different FORMAT.
* Two variables of the same name but different FORMAT in the same SCOPE will cause an error as well, because that is effectively overlapping, and those variables can't be distinguished in most circumstances.

===Identical Duplication Legal===

It is legal for a load to include one or more PCC files that have one or more VARIABLE: files that redefine the same variables. As long as they are the same format and scope, they are not considered an error. If either the scope or format differs, then the "No Overlapping" rules above are applied.

A difference in the "Explanation" text is not considered a difference in the variables significant enough to trigger an error. (In fact, the last loaded Explanation will "win" -> it is overwritten each time it is encountered for redefined but identical variables)

==Setting and Modifying a variable==

===MODIFY (LST files)===

The MODIFY token is used to set a variable that is accessible in the current SCOPE. (Note: It could be local to the current scope or any ancestor of the current scope... so modifying a global variable always just requires MODIFY)

The format of the token is:
MODIFY:V|W|X|Y=Z|Y=Z
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the current scope. Therefore, it must be either a global variable or a local variable on the current object or its parents.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Give a Player Character 2 hands:
MODIFY:Hands|SET|2

===MODIFYOTHER (LST files)===

The MODIFYOTHER token is used to set a variable that is not accessible in the current SCOPE. Beyond what is necessary in MODIFY, this token also requires the "address" of where to go before it runs the modification. Think of it as the ability to remotely place a MODIFY on another object (but revoke that MODIFY if the current object is no longer granted to the PC)

The format of the token is:
MODIFYOTHER:T|U|V|W|X|Y=Z|Y=Z
* T is the SCOPE of the object on which the resulting MODIFY (as defined by V-Z) should occur. This is closely related to the format of the object.
* U is the object or GROUPING of objects on which the MODIFY (as defined by V-Z) should occur
** Note that this does not and will not support TYPE.
* V is the Variable name for which the value will be modified
** This variable MUST be visible in the scope of the addressed object. Therefore, it is likely to be a local variable on the remote object or its parents.
** In rare cases, a remote modification of a global variable is useful. It effectively adds to that global variable only if both objects are granted to the PC.
* W is the MODIFIER to be taken on the variable (e.g. SET)
** There can be multiple actions (more below), but SET will ALWAYS be available for any format.
* X is the value related to the action. If the action is set, the variable will be set to the value in this part of the token.
** Note: The value CAN be a formula. It is possible, for example, to SET a variable to OtherVar+4 ... the system will solve that formula. For legal OPERATORS (e.g. "+") , see below.
* Y is the name of an (optional) ASSOCIATION. For now, PRIORITY (more below) is the only available association.
* Z is the value of the association.

Note: If a PRIORITY is not set it is assumed to be ZERO.

Example: Reduce the HandsRequired on all Weapons by 1:
MODIFYOTHER:PC.EQUIPMENT|GROUP=Weapon|HandsRequired|ADD|-1

Note this assumes GROUP:Weapon is in the LST line of every Weapon... PCGen can't guess or derive what a weapon is :)

==Formulas==

SET as a MODIFIER allows the use of a formula for the value. This follows many mathematical systems of using parenthesis - "(" and ")" to be precise - to group (and thus prioritize parts of the calculation).
There are a number of built in OPERATORs (see below). The system can also use functions, which use parenthesis to contain their arguments. For example:
* min(this,that)
Would return the minimum of the two variables "this" and "that".

===Built in Functions===

The system also has a series of built-in functions that work with the NUMBER format:
* abs: calculates the absolute value of the one argument.
* ceil: calculates the next integer that is >= the value of the one argument.
* floor: calculates the next lower integer that is <= the value of the one argument
* if: conditionally performs an operation. The first argument must resolve to a BOOLEAN. If true, the second argument is evaluated; if false, the third argument is evaluated.
* max: calculates the maximum of the provided two or more comma-separated arguments
* min: calculates the minimum of the provided two or more comma-separated arguments
* round: calculates the closest integer to the value of the one argument. Exactly 0.5 is rounded up.
* value: returns the value of the calculation before the current modification was started (no arguments)

Additional functions specifically related to PCGen are available:
* get: This function takes two arguments. The first is an Object type (like "SKILL"), the second is the key of a CDOMObject (like a Skill) (also in quotes)
**Example: get("SKILL","Hide")
* getFact: This function returns the value of a FACT. The first two arguments are much like the two arguments of "get", the third argument is the name of the FACT to be returned

=Legal Values=

==FORMATs==

Currently, the following are built-in formats:
* NUMBER: This is a mathematical value. Note that integer arithmetic is done if possible, so if you have an integer 6 and an integer 3, 6/3 will be an integer 2. If the 6 or 3 is a decimal value (double), then integer arithmetic was not possible and thus rounding may occur.
* STRING: A String
* BOOLEAN: A True/False value
* ORDEREDPAIR: An ordered pair of numbers. This could be an X,Y point (or in a game system more like a FACE)
* DICE: A value that takes the form AdB+C. A is the number of rolls of the die, B is the number of sides on the die, and C is the value to be added afterwords
* ALIGNMENT: An alignment object (None is default typically)

==OPERATORs==

For NUMBER, the following modifiers are available:
* +: performs addition
* -: performs subtraction (or is the unary minus operator to make a value negative)
* *: performs multiplication
* /: performs division
* %: performs a remainder function
* ^: performs an exponential calculation
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not a NUMBER)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not a NUMBER)
* >: Checks for greater than (NOTE: Returns a BOOLEAN, not a NUMBER)
* <: Checks for less than (NOTE: Returns a BOOLEAN, not a NUMBER)
* >=: Checks for greater than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)
* <=: Checks for less than or equal to (NOTE: Returns a BOOLEAN, not a NUMBER)

For BOOLEAN, the following modifiers are available:
* ==: Checks for equality
* !=: Checks for inequality
* &&: Performs a logical AND
* ||: Performs a logical OR
* !: Negates the value

For other formats, the following operators are always available:
* ==: Checks for equality (NOTE: Returns a BOOLEAN, not the compared format)
* !=: Checks for inequality (NOTE: Returns a BOOLEAN, not the compared format)

==MODIFIERs==

For NUMBER, the following modifiers are available:
* SET: This sets the variable to the value provided in the "X" parameter
* ADD: This adds the current value of the variable to the value provided in the "X" parameter
* MAX: This takes the maximum value of the current value of the variable and the value provided in the "X" parameter (i.e. min(current value, "X"))
* MIN: This takes the minimum value of the current value of the variable and the value provided in the "X" parameter (i.e. max(current value, "X"))
* MULTIPLY: This multiplies the current value of the variable and the value provided in the "X" parameter
* DIVIDE: This divides the current value of the variable by the value provided in the "X" parameter

For other Formats, the SET modifier will always be available

==GROUPINGs==

At present, the second argument of MODIFYOTHER supports 3 possible values:
* ALL
** This means the MODIFYOTHER will impact ALL objects of that type. This is useful for setting a general default for that value (e.g. you could default "HandsRequired" on weapons to 1)
* GROUP=X
** This is set by the GROUP token in LST data
* A specific object KEY

More will be added over time.

==SCOPEs==

The following are the legal scopes (not including GLOBAL):
* PC.EQUIPMENT: This is a variable local to equipment
** PC.EQUIPMENT.PART: This is a variable local to an equipment PART (or in older terms, an Equipment HEAD). Note that due to double sided weapons, Damage, CritMult and other items are generally on an equipment PART, not on the Equipment itself. Note also that this is a SUBSCOPE of the EQUIPMENT scope (so all variables in the EQUIPMENT scope can also be seen in the EQUIPMENT.PART scope)
* PC.SAVE: This is a variable local to Save (or if you prefer, Check)
* PC.SIZE: This is a variable local to the Size of a PC
* PC.SKILL: This is a variable local to a Skill
* PC.STAT: This is a variable local to a Stat

Note that all of these start with "PC." because they are impacting the current Player Character.

More will be added over time.

DYNAMIC can also be used to create a Scope and Object Type (see below)

=Examples=

==Understanding SubScope in practice==

To understand how local variables work in SubScopes, consider the following (note this is not all the variables we will use but a critical set to define the scope):
GLOBAL:NUMBER=Hands
LOCAL:PC.EQUIPMENT|NUMBER=HandsRequired
LOCAL:PC.EQUIPMENT|BOOLEAN=Penalized
LOCAL:PC.EQUIPMENT.PART|NUMBER=CritMult

The Equipment part is what the traditional system called a "head" (so a double headed axe has two "parts" in the new system - this was done since in some game systems, an item could have more than 2)

To understand how variables are available and how they get modified consider this:

Inside of a piece of equipment it could do something like:
MODIFY:Penalized|SET|HandsRequired>Hands

This would be "true" if HandsRequired on the Equipment were more than the Hands on the PC. The "Hands" variable is Global, so it is available anywhere. A Template of "Extra Hands" could do something like:
MODIFY:Hands|ADD|2
... and that adds to the Global variable "Hands".

Now consider an Equipment Modifier that was something like "Lighter Than Normal". Assume the game benefit was one less hand is required. This would look like:
Lighter Than Normal MODIFY:HandsRequired|ADD|-1
(This ignores for the moment any situation like "can't be reduced to below one".)

Note that the EquipmentModifier falls into EQUIPMENT.PART, so that it has access to the HandsRequired variable from the EQUIPMENT scope and can modify it. Note that this change could then impact the result of the BOOLEAN variable Penalized, since HandsRequired changed. Note also that "HandsRequired", when evaluated anywhere in that Equipment is now 1 less. Even if another EquipmentModifier evaluates HandsRequired, it is analyzing the variable from its parent scope, so that value on a variable from EQUIPMENT is the same in every EquipmentModifier (in the EQUIPMENT.PART scope) of that piece of Equipment.

==Understanding SubScope on Equipment==

Since variables like CritMult and Damage would be on an Equipment "part" it is important to understand how to get access to that "part" from the LST code. If a single-headed weapon wanted a larger CritMult, it would be silly to have to add a Dummy EquipmentModifier just to modify the CritMult on the part. (Much of the point of the new system is to make things more direct)

There is a token for Equipment called PART. This effectively takes 2 arguments: A number (today 1 for the primary head, 2 for the secondary head) and a full token otherwise valid on a plain object. MODIFY and MODIFYOTHER will both work in that situation, so for a piece of Equipment to set its own CritMult to 3 on its primary (and perhaps only) head, it would do:
PART:1|MODIFY:CritMult|SET|3

Note since the MODIFY is a "real" token call it uses ":" after the MODIFY which makes PART look slightly different to most tokens which use | as an internal separator. Since PART is effectively calling a "real" token, we need to use the inner ":" in order to have it pass through the same code/parsing system.

==Understanding SubScopes in relation to MODIFYOTHER==

* Note Jan 29 2018: There is a possibility of a solution for this issue that would allow MODIFYOTHER to do direct addressing of sub-objects under certain conditions

Normally MODIFYOTHER allows you to "address" another scope. For example:
MODIFYOTHER:PC.EQUIPMENT|Bastard Sword|HandsRequired|ADD|-1

However, since items like Damage and CritMult are actually on the EQUIPMENT.PART, that is different. If you consider how you would have to "address" the first head on "Bastard Sword", you would need something like:
SOMEWEIRDTOKEN:PC.EQUIPMENT|Bastard Sword|PART|1|CritMult|ADD|1

Note that this requires both a primary and secondary address. No token like this exists (at the moment). To alter items in a SubScope, you need to pass them down to the SubScope. An attempt to modify it at the Equipment level, such as:
MyAbility MODIFYOTHER:PC.EQUIPMENT|ALL|CritMult|ADD|1
... will fail (at LST load) because CritMult was on EQUIPMENT.PART, so it is not visible to the EQUIPMENT scope. Said another way, you can't universally modify all EQUIPMENT.PART variables by attempting to use that variable at a higher scope.

A more appropriate method (since this is relevant to a Feat that might alter CritMult) is to add another variable at the EQUIPMENT level:
LOCAL:EQUIPMENT|NUMBER=CritMultAdder
If all the heads then get
MODIFY:CritMult|ADD|CritMultAdder
... you can have the Feat do a MODIFYOTHER to alter CritMultAdder and the MODIFY will "pull" that into each PART:
SomeFeat MODIFYOTHER:PC.EQUIPMENT|ALL|CritMultAdd|ADD|1
So the data can be set up to only require one modification from a Feat and each Equipment Part can pull that modification down into the part. (A tangential note for those objecting that the known modifiers of CritMult select a specific weapon type: Yep, got it. That's not possible yet, so not documented yet. Suspend disbelief on the examples - Imagine a more powerful Feat for now)

==Understanding PRIORITY and processing of MODIFY and MODIFYOTHER==

When a PlayerCharacter has multiple items that attempt to modify a variable, they are sorted by two systems:
* First, they are sorted by their user priority. This is provided in the PRIORITY=Z association. Lower PRIORITY will be processed first.
* Second, they are sorted by their inherent priority. This is a built-in system that approximates mathematical priority rules. It ensures, for example, that a SET will occur before ADD if they both have a matching user priority.

Example:
Assume we had the following items:
MODIFY:Hands|MULTIPLY|2|PRIORITY=100
MODIFY:Hands|SET|2|PRIORITY=50
MODIFY:Hands|ADD|1|PRIORITY=50

The first thing is that we can tell from the PRIORITY items that the SET and ADD will both occur BEFORE the multiply (because 50 < 100). For the SET and ADD, we would need to know the inherent priority (For what it's worth, SET is 0, ADD is 3, MULTIPLY and DIVIDE are 1 and 2 since they are higher priority than ADD in traditional math). This means the SET will happen first. So the result is Hands is SET to 2, then ADD 1 to get 3, then MULTIPLY by 2 to get 6.

By controlling the PRIORITY, the data team can reorder any calculation and perform much more complicated calculations than was possible with BONUS (where movement, for example, had multiple different BONUS tokens to try to do adding before and after a multiply). This is much the same as using parenthesis in a formula in normal math, but this allows you trigger any required ordering even if the MODIFY statements are in different objects.

==Using the previous value in a more complex calculation==

Take a modification that, for example, wishes to perform a calculation such as: "Reduce the value by 1, but to no less than 1". We could write this as two MODIFY statements, but that is probably undesirable for a few reasons:
* If the book states it as one sentence, it would be preferable to write it as one MODIFY
* It makes the calculation a bit harder to follow as to what is going on
* If the items are different PRIORITY, there is a risk of another dataset attempting to (or accidentally) putting something in between those two steps, resulting in an incorrect calculation.

Therefore, we need a method of drawing on the current value in a formula. To do that we use the value() function. The calculation above then becomes:
max(value()-1,1)
...and we can use it in a MODIFY as such:
MODIFY:HandsRequired|SET|max(value()-1,1)

Note that we are using SET here - if a formula is used, it is likely to be best done as a SET, since that will be much more clear than calculating a formula and then immediately applying another MODIFICATION. Note that some modifications may also prohibit a formula depending on their underlying behavior.

=Global Modifier File=

There is a new file type that is defined in the PCC files with GLOBALMODIFIER.

This file allows for a centralized (and clear to the data user) location for global modifications (don't hide things on stats or elsewhere anymore). This supports MODIFY for global variables (if you want to start all PCs with 2 hands for example) or MODIFYOTHER for local variables.

==GLOBALMODIFIER (PCC File)==

Format:
GLOBALMODIFIER:x
* x is the file to be loaded as a GLOBALMODIFIER file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

=Advanced Topics=

==Dynamic Objects and Scope==

This set of features allows the data to supplement the defined SCOPEs with new object types not originally comprehended in PCGen. Within a Dynamic Scope, the data can produce objects.

Warning: Attempts to define a SCOPE that overlaps with existing objects (e.g. LANGUAGE) will not produce a warning today but will get you in trouble long term. It won't work in the way you would like it to work (it won't attach a SCOPE to that object type), so just avoid those overlapping names. Basically if you can CHOOSE it, or use it with FACT, don't DYMAMICSCOPE it.

===DYNAMICSCOPE (DATACONTROL LST)===

The DYNAMICSCOPE token (used in files referred to by the DATACONTROL: token in PCC files) can define a Dynamic Scope.

DYNAMICSCOPE:x

*x is the new scope
**Any argument to DYNAMICSCOPE becomes a legal "scope" in LOCAL: in the variable definition file
**Limit of one new scope per line. (No delimiter on this token)
*Like other LST tokens, this is case insensitive, but usage in later tokens is always capitalized, so I suspect the data standard should be ALL CAPS

===DYNAMIC (PCC File)===

In order to create objects within the defined DYNAMICSCOPE, DYNAMIC: becomes a new token legal in PCC files. It's format is:

DYNAMIC:x

* x is the file just as in tokens like DEITY or DOMAIN
* INCLUDE/EXCLUDE on the line are are legal.

Within the Dynamic file, the Scope of the new objects being created MUST appear as a prefix token on the line. For example, if there was a DYNAMICSCOPE:VISION, then the following would be legal in a DYNAMIC file:
VISION:Darkvision
VISION:Low-Light Vision

* The "Token Name" (what appears before the ':') is the DYNAMICSCOPE name.
* These are created as basically hollow, simple objects.
* These objects CAN contain variables.
(It is the intent to allow these at some point to also have MODIFY* tokens, but that is not currently supported)

===GRANT (LST files)===

A Dynamic object MUST be granted in order to have its local variables modified or to have its modifiers have an impact (just like any other object)

GRANT:x|y

* x is a DYNAMICSCOPE
* y is the name of the dynamic object.

===Export===

Dynamic items can be exported in Freemarker using the

<#list pc.dynamic.x as y>

* x is the name of the DYNAMICSCOPE
* y is the local variable within the #list that the output team wants to use to access the dynamic object
* It is likely for ease of use by the output team, x and y will be identical

For a complete example using output, see below

===Example===

DATACONTROL: file:
DYNAMICSCOPE:VISION

DYNAMIC: file:
VISION:Normal
VISION:Darkvision
VISION:Low-Light Vision

VARIABLE: File:
LOCAL:VISION|NUMBER=Range

GLOBALMODIFIERFILE:
MODIFYOTHER:PC.VISION|ALL|Range|Set|60

Sets a "default" range for any VISION, to avoid setting on each VISION object

RACE LST file:
Human <> GRANT:VISION|Normal

Export:

<#list pc.dynamic.vision as vision>
${vision.name} ${vision.val.range}
</#list>

(this ignores converting the range to local units and all that, but you get the idea)

==ARRAY Format==

In addition to the base formats, there is an ARRAY[x] format:
ARRAY[x]
* x is an existing valid Format
** x CANNOT be ARRAY (multi-dimensional arrays not supported
** Just to be precise, yes DYNAMIC formats are legal

(Note: This may get renamed - we need to decide on a few things for how this will work vs other types of collections)

===Modifiers===

There are two valid MODIFIERs for ARRAY:
* ADD: This adds a value to the ARRAY
** Mathematically, ARRAY acts as an ordered set of objects. Therefore, if you add an item to an array more than once it is ignored.
* SET: This sets the contents of the ARRAY

==Custom Functions==

Within certain game systems, there are calculations that are likely to be repeated. In order to simplify the calculation of these items, there is an ability to create custom functions.

A function can have zero or more arguments. The number is defined by what appears in the VALUE token (see below).

Note that the function actually has to be *used in the data* (not just defined) for most errors to be caught (since the system can't guess at overall context before a function is used and it would be possible to write a function that would work in more than one FORMAT).

===Function (DATACONTROL LST)===

FUNCTION:x
* This token must appear as the first token on a line in a DATACONTROL file.
* x is the name of the function. This name must starts with a letter, e.g. A-Z, additional legal characters are 0-9 and _ (none of those additional legal characters as the first character)
** x must not contain spaces, no other special characters.
** Function names are case insensitive (as many other things in LST files)

A Function MUST also contain a VALUE, which defines how the function is calculated.

If a FUNCTION appears more than once, the other characteristics (VALUE) must be identical.

===Value (DATACONTROL LST)===

VALUE:x
* Must appear as an additional token on the line of a FUNCTION: in the data control file
* x is a valid formula.
** This means it must have valid variable names, valid function names, matching parenthesis, etc.

In addition to all of the built-in functions (e.g. if, ceil, round), two additional items can be used:
* the arg(n) function (see below).
* Any previously defined FUNCTION. This MUST appear in the file before the current Function or in a file processed before the current file.

===arg (Function)===

The arg function is a "local" function to the VALUE: part of a FUNCTION (it can not generally be used in LST files). The arg function takes one argument. It MUST be a Integer >= 0.

arg(x)
* x is an integer number (zero-indexed)

When arg(x) is used in value, the function pulls from the arguments that were provided to the original function in the data.

===Using a Function===

When a function is used in LST data, it is called by the name provided in the FUNCTION token. It requires a certain number of arguments. This exactly matches the integer one greater than the highest number of the arg(n) function provided in the VALUE part of a FUNCTION (In computer science terms, the arg(n) function is zero-indexed). The provided arguments can be any legal formula, so:

d20Mod(INT)
d20Mod(INT+4)

...are both perfectly legal.

===Example===

The modification calculation in most d20 games is fairly easy to calculate, but is repeated very often. It basically takes (ability score - 10)/2.

Here is how we would define a new function d20Mod in the DATACONTROL file:
FUNCTION:d20Mod VALUE:floor((arg(0)-10)/2)

This would then be used in LST data as:

d20Mod(n)

The system will catch both of these as errors:

d20Mod()
d20Mod(14,5)

These have too few or too many arguments, respectively. It would also catch:

d20Mod("mystring")

...as an error (wrong format - requires a Number, found a String).

The n is substituted where "arg(0)" appears in the "VALUE"... so a statmod would be as easy as:

d20Mod(INT)

or some such... Direct values can also be used, e.g.:

d20Mod(10)

...would return 0.

==Performance Considerations==

It is possible to write two identical modifiers that perform the same net calculation on a PC. For example:
MODIFY:Age|ADD|2
MODIFY:Age|SET|value()+2

...perform the same calculation (adding 2 to Age).

Why have ADD at all? Answer: These are NOT AT ALL equivalent in memory use or speed/performance.

The ADD (Since it's adding an integer) occupies approximately 40 bytes of memory and is an extremely rapid calculation: If it took 200 nanoseconds I'd be surprised. This is because ADD is using a number. If it was a formula, even 1+2, it would load the formula parser. This special case on constants allows the modification system to modify a value without loading a much larger infrastructure, and this is valuable as a large majority of our calculations are simple modifications of values.

The SET shown above (since it has an operator) loads the formula system, and thus may occupy 500 (or more) bytes of memory (>10x), so it could take 100 times as long to process as the ADD (it also took longer during LST load).

Thus: When possible the data standards should be written to use numeric modifiers and no operator. (Think "static" modifiers ... that only use numbers) rather than a formula.

==Lookup and Tables==

For some situations, it makes more sense to have a lookup table rather than attempting to have a set of tokens or calculations perform processing. This also allows for much cleaner translation from books when the books are using tables to organize information.

The Table file format is designed to allow you to use a spreadsheet program to modify the tables and export the table in CSV format. While we don't strictly follow full CSV rules, as long as you avoid embedded quotes and embedded return carriage/line feed, it should be safe.

More than one table is allowed per file (to avoid sprawl and help simplify modification/management of tables).

===TABLE (PCC File)===

TABLE:x
* x is the file to be loaded as a TABLE file
* Limitations: It does not support PRExxx or INCLUDE/EXCLUDE.

====Table LST file format====

File formatting considerations:
* Trailing commas will not impact any TABLE, and will be ignored
* Blank lines will be ignored. This includes lines with commas but without content. (allows tables to be separated by blank lines or lines of all commas)
* Attempts to use embedded line breaks or embedded quotes may or may not be supported by the parsing system, but certainly aren't supported for purposes of PCGen.
* Lines that have the first cell starting with "#" are considered comment lines and will be ignored. This will likely need to include if the first cell is escaped in quotes, just for protection from any tools that quote by default.
* Since the CSV format ignores leading/trailing spaces, one can easily have a spaced out version that is a table in a text editor if folks don't want to edit in a spreadsheet. (Having someone write a "PrettyTable" ... a trivial perl or python program to do that for the data team seems fairly easy)

A TABLE starts with STARTTABLE:
STARTTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the STARTTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If a STARTTABLE is encountered when another Table is active, an LST load error will occur.

A TABLE ends with ENDTABLE:
ENDTABLE:x
* x is the name of the Table. Most characters are allowed, avoiding quotes is advisable.
* Allowing Trailing commas allows the ENDTABLE lines to have blank cells where the rest are the table contents... we don't want to complain on minor things that may tools will do.

If an ENDTABLE is encountered without a STARTTABLE, an LST load error will occur.

A Table must have a minimum of 3 rows, NOT counting the STARTTABLE and ENDTABLE tokens.

The First row:
x,x
* x is the column name. These are always the first line of the table after STARTTABLE:

Any column which has data MUST have a name.

The Second row:
y,y
* y is the format for the column. Any column that was named must have a FORMAT.

Additional rows:
z,z
* z represents contents of the table. These must be in the FORMAT provided for the appropriate column.

====Example====

STARTTABLE:Carrying Capacity,
Strength,Capacity
NUMBER,NUMBER
1,10
2,20
...
29,1400
ENDTABLE:Carrying Capacity,

===TABLE[x] Format===

When a TABLE is created, it implicitly picks up a FORMAT based on the FORMAT of the first column of the Table.
Unlike basic FORMATs shown above, TABLE has a SUBFORMAT (in brackets).

TABLE[x]
* x is the SUB-FORMAT of the TABLE (this is the format of the first column of the Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. It certainly is prohibited to create a TABLE[TABLE[x]], but other situations the behavior may not be prohibited by the code (but the behavior is also not guaranteed)

===COLUMN[x] Format===

When a Column needs to be referred to in the data, it has a special format as well:

Unlike basic FORMATs shown above, COLUMN has a SUBFORMAT (in brackets).
COLUMN[x]
* x is the SUB-FORMAT of the COLUMN (this is the format of the data that appears in that column in a Table)
** x must be an otherwise valid FORMAT, and it is advised to avoid any FORMAT that has a SUB-FORMAT
*** This limitation may be inconsistently enforced. Such behavior may not be prohibited by the code (but avoiding bad behavior is also not guaranteed in a situation where embedded brackets occur)

===Using Tables===

With MODIFY, a TABLE[x] variable doesn't look any different. It's still:
MODIFY:TableNameVar|SET|"SomeTable"

"SomeTable" is still a string to a naive reader, but what the system will do at that point is make sure that SomeTable can actually be interpreted as a TABLE[x]... rather than just treating it as a String. Similar for column names when they are encountered (see below).

In that way, the sub variable will allow multiple tables to be swapped in, AS LONG AS they are the right format. If the format is not compatible, then all bets are off and you'll get an error.

===lookup() Function===

The major consideration is how to do the lookup in data. We can borrow some inspiration from things like Excel and Google Docs:

lookup(X,Y,Z)

* X is a table. This must resolve in the formula system to a valid table. For now, a get(...) function may be required.
* Y is the lookup value in the table. It must match the format of the first column in the table. At this time, it does an EXACT match.
* Z is the target column. This must resolve in the formula system to a valid table column. It must appear in the table named by the contents of the first argument of lookup.

Example: Max load calculation would be something like:
lookup("Carrying Capacity",floor(StrScore),"Capacity")*SizeMult
We have to do the floor due to the exact name of the lookup

Note: The format of both TABLE "Carrying Capacity" and COLUMN "Capacity" can be derived by PCGen internally and thus are no longer required to have a get (current as of Jan 30, 2018)

Note that this function allows some runtime errors, but catches a lot of possible issues. It is still possible to do a lookup on a Table that doesn't contain a specific column of the given name or format, simply because those items can be variables resolved at runtime. For example, if the third variable is COLUMN[NUMBER], we can check at load that the TABLEFORMAT has columns of FORMAT=NUMBER, but we can't guarantee the variable will resolve to a name actually in that table... that will have to wait for a runtime error. That's life, but it does allow us to know the formats at load time, so we do get a lot of error checking in the context of the usage, if not the exact table and column names.

Note you can do some really dynamic behaviors to have folks override game mode behaviors. For example, one neat item would be that we could alter a query for dice steps to extract the table name into the Global Modifier file:
MODIFY:DiceStepTable|SET|"Dice Step"
lookup(get("TABLE[NUMBER]",DiceStepTable),BaseDamage,get("COLUMN[NUMBER]",lookup("Step Column",CurrentSize-BaseSize,"Name")))

Note: In both cases here, the "get" is shown for completeness. In the case of DiceStepTable - it MAY be required or prohibited - it depends on what the Format is of the "DiceStepTable" variable. If it's TABLE, then the get is not necessary (and is actually prohibited). The example shown above assumes DiceStepTable is a String. The better design would be for it to be a TABLE (specifically TABLE[NUMBER]). For the "get" after the lookup, that is required - the lookup value will return a STRING, and as such, will needs to be converted to a COLUMN.

Now, if someone needs different dice steps for some reason... they can override the DiceStepTable variable and it will look up in a different table... so expansions could truly be expansions and not have to reach back into core PCC files in order to change key behaviors.

Consider also that spell tables can be shared across classes, or if a subclass alters the default, just create a new table rather than having to BONUS/MODIFY all the numbers in a table with adding and subtracting...

==To be completed==
* getOther
* Channels

=Material Changes=

==lookup function simplification==

Effective January 30, 2018, tables and columns can be referred to directly in lookup(). This is actually a "broad" change to how strings are interpreted. The system - when possible - fully asserts a FORMAT, and first attempts to convert the given String per the rules of that FORMAT. This also means an assignment of an ALIGNMENT, by key, will not require a "get":
MODIFY:Alignment|SET|"LE"
...will be fully legal. (This assumes Alignment is FORMAT: of ALIGNMENT)

Working Projects

2018-11-10T22:03:12Z

Tom Parker: /* What is the solution? */

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

Good places to start: RaceFacade, DeityFacade, HandsFacade, GenderFacade

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Note: Strictly the new method in Step 1 of Phase 1 is not necessary, although it may help you identify places to look. Longer term it may make sense either to not do the new method and place things directly into the initComponents method, or to do the two phases below and then inline those initializeStaticModel methods at the end. Either way, the target here is the createModels, restoreModels and storeModels methods.

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

=UI Consolidation=

==What is the situation today?==

QualifiedFilterHandler is (to within generics) identical between Race, Template, Domain, and potentially other tabs.

The same is true of the XInfoHandler methods in various tabs.

The same is true of availPanel and selPanel, built in the initializers of the classes for various tabs.

==Where do I see this stuff?==

pcgen.gui2.tabs.* QualifiedFilterHandler is, today, an inside class

==What is the solution?==

# Extract one of them into its own class, make them all shared, slightly expanding generics if necessary.
# For the XInfoHandler classes, you may have to construct it with a few things, but still consolidation is possible
# For the Avail/Sel panel items, create two new classes, the boxes, glue, addAction and lots of other stuff can be isolated from each of the tabs.

Note: Each of these new classes probably belongs in elsewhere... e.g. pcgen.gui2.util (or in pcgen.gui2.handler, etc) not in tabs.

Working Projects

2018-11-10T21:59:24Z

Tom Parker: /* What is the solution? */

Working Projects

2018-11-10T21:58:43Z

Tom Parker: /* UI Consolidation */

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

Good places to start: RaceFacade, DeityFacade, HandsFacade, GenderFacade

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Note: Strictly the new method in Step 1 of Phase 1 is not necessary, although it may help you identify places to look. Longer term it may make sense either to not do the new method and place things directly into the initComponents method, or to do the two phases below and then inline those initializeStaticModel methods at the end. Either way, the target here is the createModels, restoreModels and storeModels methods.

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

=UI Consolidation=

==What is the situation today?==

QualifiedFilterHandler is (to within generics) identical between Race, Template, Domain, and potentially other tabs.

The same is true of the XInfoHandler methods in various tabs.

The same is true of availPanel and selPanel, built in the initializers of the classes for various tabs.

==Where do I see this stuff?==

pcgen.gui2.tabs.* QualifiedFilterHandler is, today, an inside class

==What is the solution?==

Extract one of them into its own class, make them all shared, slightly expanding generics if necessary.
For the XInfoHandler classes, you may have to construct it with a few things, but still consolidation is possible
For the Avail/Sel panel items, create two new classes, the boxes, glue, addAction and lots of other stuff can be isolated from each of the tabs.

Working Projects

2018-11-10T21:54:41Z

Tom Parker: /* Filter Consolidation */

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

Good places to start: RaceFacade, DeityFacade, HandsFacade, GenderFacade

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Note: Strictly the new method in Step 1 of Phase 1 is not necessary, although it may help you identify places to look. Longer term it may make sense either to not do the new method and place things directly into the initComponents method, or to do the two phases below and then inline those initializeStaticModel methods at the end. Either way, the target here is the createModels, restoreModels and storeModels methods.

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

=UI Consolidation=

==What is the situation today?==

QualifiedFilterHandler is (to within generics) identical between Race, Template, Domain, and potentially other tabs.

The same is true of the XInfoHandler methods in various tabs.

==Where do I see this stuff?==

pcgen.gui2.tabs.* QualifiedFilterHandler is, today, an inside class

==What is the solution?==

Extract one of them into its own class, make them all shared, slightly expanding generics if necessary.
For the XInfoHandler classes, you may have to construct it with a few things, but still consolidation is possible

Working Projects

2018-11-10T21:52:48Z

Tom Parker: /* What is scope of work? */

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

Good places to start: RaceFacade, DeityFacade, HandsFacade, GenderFacade

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Note: Strictly the new method in Step 1 of Phase 1 is not necessary, although it may help you identify places to look. Longer term it may make sense either to not do the new method and place things directly into the initComponents method, or to do the two phases below and then inline those initializeStaticModel methods at the end. Either way, the target here is the createModels, restoreModels and storeModels methods.

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

=Filter Consolidation=

==What is the situation today?==

QualifiedFilterHandler is (to within generics) identical between Race, Template, Domain, and potentially other tabs.

==Where do I see this stuff?==

pcgen.gui2.tabs.* QualifiedFilterHandler is, today, an inside class

==What is the solution?==

Extract one of them into its own class, make them all shared, slightly expanding generics if necessary.

Working Projects

2018-11-10T21:48:34Z

Tom Parker:

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

Good places to start: RaceFacade, DeityFacade, HandsFacade, GenderFacade

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

=Filter Consolidation=

==What is the situation today?==

QualifiedFilterHandler is (to within generics) identical between Race, Template, Domain, and potentially other tabs.

==Where do I see this stuff?==

pcgen.gui2.tabs.* QualifiedFilterHandler is, today, an inside class

==What is the solution?==

Extract one of them into its own class, make them all shared, slightly expanding generics if necessary.

Working Projects

2018-11-10T21:08:47Z

Tom Parker: /* Eliminate Facades */

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

Good places to start: RaceFacade, DeityFacade, HandsFacade, GenderFacade

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

Working Projects

2018-11-10T20:13:33Z

Tom Parker:

=A COMMENT OVERALL=

It is very helpful for us reviewing the code if you check these in in small pieces and not one massive bulk update. That is one reason why I suggest incremental changes.

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

=Move UI to Dependency Injection model=

==What is the situation today?==

Currently, there are a set of models in the UI that are swapped out for each character. These are done passively on swap, and force many items to have knowledge of CharacterFacade.

==Why do we do this?==

# We only want to build UI items once, and swap the model underneath

This design has a few limitations that we want to get rid of:
# We want to move toward a "dependency injection" model
# We want to lower dependencies on runtime calls up and down a stack (there are classes in a tangle that DI can eliminate)

==Where do I see this stuff?==

# pcgen.gui2.tabs for the most part

==What is the solution?==

Convert these items to dependency injection and eliminate the model swapping effects in the UI.

==Is an example shown today?==

No.

==What is scope of work?==

Phase 1:

# Start with code/src/java/pcgen/gui2/tabs/CharacterInfoTab.java ... Add a method: public void initializeStaticModel(Supplier<CharacterFacade> supplier);
# Add an empty version of the method above to all classes that implement CharacterInfoTab
# pcgen.gui2.tabs.InfoTabbedPane needs to be modified. It:
## Owns the Supplier
## Must call the method above in initComponent()
## May need to initialize the Supplier object and the value of the Supplier in its constructor (depending on how the Supplier is written)
## Note: Null must be a legal return value for the supplier, and it should be initialized to null.

Phase 2:

# For each of the classes where initializeStaticModel was added, look at the following method: public ModelMap createModels(CharacterFacade character)
# For each of the values of a put statement adding to the ModelMap, do the following:
## Evaluate if that subclass needs the CharacterFacade. If so, provide that subclass the supplier in the constructor to that object and alter the class to use the Supplier.
## Delete the put, rather taking the construction of that object (which now uses the supplier) and place it into initializeStaticModel.
## Delete the "install" in restoreModels(...)
## Delete the "uninstall" out of saveModels(...)
## Delete the "uninstall()" method on modified subclass
## Take the items from the "install()" method on the modified subclass and add those to initializeStaticModel (correcting for the difference in location)
# It is almost certain you will run into side effects where the Supplier will need to be passed into a constructor at some point - this is the idea, and part of DI.
WARNING: For your own sake, do these in small pieces, one target at a time. You will rapidly find a few (QualifiedTreeCellRenderer comes to mind) that will have a wide ranging impact.

Working Projects

2018-11-10T19:24:49Z

Tom Parker:

=Token Rebuilds to Interface=

==What is the situation today?==

Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL database.

This process today uses things like AbstractObjectContext (which inside it has an ObjectCommitStrategy) in order to do this level commitment process.

==Why do we do this?==

# Primarily, if there is a token that has a bad syntax, we don't want a partial data structure to get into the core code. This causes a tremendous amount of "what if" scenarios in our code that means folks (a) write a ton of error checking code (b) catch and silently consume errors
# We want to catch LST errors at load and report that ASAP to the data team. If they have to actually USE a token in order to make sure it is valid, that is a HUGE testing burden on the data team.

This commit process has been VERY valuable (we want to keep it!), but also challenging. There are a few limitations we want to get rid of:
# This forces us to only do things that ObjectCommitStrategy knows about. There is a huge centralized code dependency (high workload, fragile)
# This forces us to remember never to write to a specific object, but always through the Context (annoying, fragile)
# This forces a design where "everything is a CDOMObject", which creates a form of demi-god object (PlayerCharacter is still our "God Object" ;)

==Where do I see this stuff?==

# The tokens are in plugin.lsttokens
# The contexts are in pcgen.rules.context
# Some abstract items are in pcgen.rules.persistence.token

An actual example of how the token loading works is in pcgen.rules.persistence.TokenSupport.processClassTokens(...)... that will return success or failure and capture any messages in case of failure into a set of parsed messages.

The actual commit/rollback process can be seen in pcgen.persistence.lst.LstUtils.processToken(...)

==What is the solution?==

The best way to get around this is to "properly" use reflection, which was really not a thing comfortable for many developers in about 2006 when we started down the token path. Thankfully this can be abstracted into a library, so the reflection is well hidden from most users/developers.

The reflection code is StagingInfo, and it exits in our Base PCGen library (pcgen-base repository) in pcgen.base.proxy.

==Is an example in the code today?==

Yes. This staging is used for a limited number of tokens and you can find the process in pcgen.rules.persistence.TokenSupport.processInterfaceToken(...)

==What is scope of work?==

At this point, some folks may be thinking things like "Hey - I can have a getAlignment and putAlignment method on Deity.class!". That may be literally true, but there are a few considerations to that:
# One step at a time
# There are things outside the scope of this description that are also changing and make the first step as described here valuable and a dramatic step to completely eliminate the abstraction in CDOMObject to be potentially undoing other things we value

The scope of work here is to change indirect calls through an object context, e.g.:
context.getObjectContext().put(deity, ObjectKey.ALIGNMENT, al);
...into direct calls on the object, e.g.:
deity.put(ObjectKey.ALIGNMENT, al);

This directly meets requirements #1 and #2 above.

In order to do this a few things need to happen:
* Read and write interfaces need to be defined, so they can be passed to a StagingInfo
It is my belief that the interface would work best right now if the scope was a single behavior. So the put and get just for ObjectKey.* would be processed separately.
* The tokens that are using ObjectContext to use those methods need to call items directly on the object
* A method needs to be defined that will appropriately limit usage of the object
*: This may require a bit of explanation. Let's use plugin.lsttokens.deity.AlignToken. Today, this keys directly off the object, Deity, using CDOMPrimaryToken as the interface (the parent interface CDOMToken actually defines how TokenSupport will process that token).
*: If we create interfaces ReadObjectKey and WriteObjectKey, those can be reused across all of the same types of methods, but it places us in a situation where we have lost the ability to enforce ALIGN:x only works on a Deity. So there is a balancing act here:
*: We probably don't want to have interfaces directly for each type of object (readDeity and writeDeity), as that creates interface proliferation (and we want to get away from that level of specific behavior anyway)
*: We probably want to think about how we can have tokens actually work on Dynamic objects - this relates to (A) above... the dynamic objects are all the same class, and thus a different validation method is necessary than one based on class or the interfaces implemented by that class. (see pcgen.cdom.inst.Dynamic)
* It is likely that a new interface beyond CDOMPrimaryToken / CDOMInterfaceToken needs to be defined. It is possible CDOMInterfaceToken could be enhanced with a default method that always passes, but could be overridden with a method that would check for legality from #3 above in cases where it is necessary.

==What is one possible path?==

WARNING: Any mixed tokens that still use the context probably don't want to be converted, so two of these actions will be reverted at the end

# Take the import for ObjectKey out of ObjectCommitStrategy and find the errors
# Delete all the methods that are then errors from those methods being removed from the interface
# A whole bunch of tokens now have errors. Those need to be checked. If they are just a simple use of the interface then:
## changed to direct calls rather than through the objectcontext
## converted to the new load method (not CDOMPrimaryToken)
## Have the limit from #3 (limit where it is legal) added
## If they are complicated (FavoredClass), then I'd leave as-is for the moment.
# Revert the changes made in #1 and #2

Lather, rinse, repeat for ObjectKey, IntegerKey, ListKey, StringKey, FactKey, FactSetKey

You can skip VariableKey, FormulaKey, MapKey for now.

=Eliminate Facades=

==What is the situation today?==

There are a series of Facades (e.g. RaceFacade) that provide an interface on the core object for use by the UI.

==Why do we do this?==

# Primarily, the facades are isolation of behavior between the core and the UI.

This design has a few limitations that we want to get rid of:
# This forces the UI awareness back into the core, forcing the core to have methods from the Facade Interfaces it doesn't really "want" (and are only there to serve the UI)
# This is very design-specific. Every behavior (Race has Hands) is a game-dependent piece of information that makes PCGen less flexible. While we primarily handle d20, it still is a tight tie that makes it hard to get the UI dynamic to the data.
# This is removing a level of indirection that is necessary preparation for later work that will completely change how we access information in the UI (see below)

==Where do I see this stuff?==

# The facade interfaces are in pcgen.facade.core

==What is the solution?==

For now, this step is to delete the interfaces and directly use the objects. The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes and more. PC Alignment is now controlled through a channel... but that setup is more than this step is asking at the moment.

==What is scope of work?==

# Delete the WhateverFacade and directly use the object to get information.

=Create Channels/Delete Facets=

==What is the situation today?==

There are a series of Facets that store the primary information for the model storing the PC. These are in pcgen.cdom.facet

==Why do we do this?==
# The Facets are small containers for specific sets of information about a PlayerCharacter. These were part of the process of reducing the "god object" behavior of PlayerCharacter.
#: In a larger picture, Facets are going away. There are different types of facets, but in general, the new formula system can take over most of the behaviors of facets. (There are exceptions we will eventually address).

This design has a few limitations that we want to get rid of:
# The facets are still operating in a model where the UI has to go order the core to do things. This is *really* challenging for things that have interacting behaviors. Gender, for example, is usually PC controlled, but there are some (cursed) magic items that can override it. This really means our solving system (the formula system) needs to be UI- AND object-aware, and we want it to do the resolution of what the current situation is on a PC.
# We want to pass control of ALL of these items that touch the new formula system over to the data. Eventually, the data should know the variable is "PCsAlignment" and then the HTML (or whatever) that displays the PC should be told it's "PCsAlignment". In this way, the variable name from the data can match the HTML and NO code modifications are necessary to change what or how it is displayed. This completes a significant portion of our "separation of duties" to reduce the amount of code change necessary for minor items in game systems, and allows us to focus on larger issues.

==Where do I see this stuff?==

# The facets are in pcgen.cdom.facet, more specifically the ones relevant here are mostly in pcgen.cdom.facet.model

==What is the solution?==

The strategic direction is to use channels. See pcgen.output.channel

==Is this shown today?==

Yes. PC Alignment is now controlled through a channel. You can see some compatibility work (aka temporary code) in pcgen.output.channel.ChannelCompatibility

==What is scope of work?==

Step 1: Move from using the facet to using the new formula system, while Enabling Data to control the channel/variable. This actually takes some interacting work with data.
# We need to set up a system that tells us whether we (code) control the variable name for an item or whether data has asserted IT controls the name. These are called "CodeControls" (see pcgen.cdom.util.CControl for the names, such as ALIGNMENTINPUT)
# If we control the name, we need to create the channel. See SourceFileLoader.defineBuiltinVariables for how alignment works.
# Depending on the complexity, we may need ChannelCompatibility items. Arguably, the Alignment work is not complete, but it is at least functional.
# Delete the facet. There may be other facets using the one you deleted. If so, you will need something like: In PlayerCharacter.doFormulaSetup, you will need soemthing like: VariableChannelFactoryInst.watchChannel(id, "DeityInput", activeSpellsFacet);
If there is a priority [a number] when the listener is set up, you will need to use that as well - please see the other watchChannel method in VariableChannelFactoryInst
# If there is a new format involved (things like DEITY), then you probably need to work with the data team to have that variable defined in all data modes OR you need some method for figuring out where it is/is not legal.

Step 2: Unwind most of CharacterFacade (a bit of a concentrated set of behavior), and reduce the breadth of usage of the pcgen.facade.core.CharacterFacede object within the UI. (.getCharID() usage is good, others to be eliminated)

Note: Step 2 in some cases may be VERY hard. Alignment is an example, where changing Alignment can have side effects. We don't currently have a great design for that scenario, in that checking for qualification of the alignment, checking EX_CLASS, etc. are not behaviors we yet have a design to handle in the formula system.

Working Projects

2018-11-10T19:17:04Z

Tom Parker: /* What is one possible path? */

Working Projects

2018-11-10T19:16:15Z

Tom Parker: /* What is scope of work? */

Working Projects

2018-11-10T19:12:55Z

Tom Parker:

Working Projects

2018-11-10T19:10:05Z

Tom Parker: Created page with "=Token Rebuilds to Interface= ==What is the situation today?== Our loading tokens are using a form of commitment process (evaluate then commit or rollback) much like a SQL d..."

Code

2018-11-10T18:36:40Z

Tom Parker: /* Coding */

{| align="right"
| __TOC__
|}

=Introduction=
Welcome to the Wiki section for the Code Monkey team!

=Mission Statement=
'''TBA'''

=Resources=
* [[Developers Meeting]]
* [[Dev Meeting Log 20090429]]
* [[Dev Meeting Log 20090815]]
* [[Dev Meeting Log 20090911]]
* [[Dev Meeting Log 20091017]]
* [[Dev Meeting Log 20091114]]
* [[Dev Meeting Log 20120309]]
* [[Dev Meeting Log 20120429]]
* [[Dev Meeting Log 20121103]]
* [[Dev Meeting Log 20140627]]

==For New Starters==
* [[Joining the Code Team]] General steps to take to prepare yourself
* [[Code Monkey Introduction]]

==Developer Environment==
* [[Basic Developer Setup]]
* [[Git Setup]]
* [[Subversion Setup]]
* [[Merging]] with SVN
* [[Building PCGen]], also see [[Gradle]]
* [[Continuous Integration]]
* [[Autobuilds for Libraries]]
* [[launch4j]]

==Coding==
* [[Coding Standards]]
* [[Deprecating a Token]]
* [[Explanation of the Code Base]]
* [[PCGen Code Exploration Almanac]]
* [[Graph Theory]]
* [[Logging in the Code Base]]
* [[Unit Testing]]
* [[Removing Libraries]]
* [[Unnecessary Code Detector]]
* [[Working Projects]]

=Completed Sub Projects=
* [[Ability Object TODO List]]
* [[LST_Editor_Verification]]

=Open Sub Projects=
* [[UI HTML Extraction]]
* [[Internationalization]]
* [[CDOM]]
* [[UI Overhaul]]
* [[LST_Editor]]
* [[Major Code Projects]]
* [[Template Engine]]
* [[Enhanced Networking Features]]

=Active Team Members=

===[[Explanation of Teams#Silverback|Silverback]]===
* Connor Petty

===[[Explanation of Teams#Second|2nd]]===
* TBA

===[[Explanation of Teams#Chimp|Chimp]]===
* [[Tom Parker]]
* [[Stefan Radermacher]]

===[[Explanation of Teams#Gibbon|Gibbon]]===
* Jayme Cox

===[[Explanation of Teams#Tamarin|Tamarin]]===
* [[Andrew Wilson]]
* [[Martijn Verburg]]
* [[Connor Petty]]

===[[Explanation of Teams#Lemur|Lemur]]===
* [[Eddy Anthony]]
* Per Christian Henden
* Thomas Cooper
* [[Tod Milam]] - Mac Developer
* [[Kim Winz]]
* Eitan Adler

===[[X-Terminator Mark II]]===
* [[Greg Bingleman]] - [http://www.codemonkeypublishing.com Code Monkey Publishing]

===Other===
* Brad Stiles, SVN advice
* Jason Buchanan, Advice on old code
* [[Jonas Karlson]], Advice on old code

=Passive Team Members=

* B. K. Oxley (Provides IntelliJ licenses)

=Inactive Team Members=

===[[Explanation_of_Teams#2nd|2nd]]===
* Devon Jones

===[[Explanation_of_Teams#Chimp|Chimp]]===
* Aaron Divinsky

===[[Explanation_of_Teams#Tamarin|Tamarin]]===
* Andriy Sen
* [[Kevin Fernandes]]
* [[Koen Van Daele]]
* [[Joe Frazier, Jr.]]
* Per Christian Henden

===[[Explanation_of_Teams#Lemur|Lemur]]===
* Bill Neumann
* Dan Parks
* [[Eduard Martinescu]]
* Eric Jarman
* Jasper Spaans
* Julio Esslinger Viegas
* Peter Barker
* [[Rick Ryker]]
* Thomas Clegg
* Tony Lavalle
* Hades Lucifer [7/20/10]
* Josh Johnston [7/20/10]
* MotorViper [7/20/10]
* [[Steven West]] [7/20/10]
* [[Alec Ross]] [7/20/10]
* [[Tir Gwaith]] [7/20/10]

===Other===
* [[Andargor]], Suggestions
* [[Bryan McRoberts]], Advice on old code
* Kurt Wimmer, Suggestions
* Walter Duncan, Held the old pcgen_autobuild user

UI HTML Extraction

2018-03-26T21:19:48Z

Tom Parker: Created page with "{| align="right" | __TOC__ |} =Purpose= This project externalizes a number of items in the PCGen UI into something that is outside the code. This WILL require some dis..."

{| align="right"
| __TOC__
|}

=Purpose=
This project externalizes a number of items in the PCGen UI into something that is outside the code.

This WILL require some discussion with members of the team (code, data, and output, release).

==Reasoning==

We are moving this out of the code because they are closely tied to the data. By moving them out, we can accomplish a number of things:
# Have different display in different game modes, depending on the content of an item
# Have the display be able to be updated when data changes are made, rather than requiring code/data co-dependence
# Leverage a common system for output
# Prepare for a future UI change by extracting behavior that will be needed regardless of the presentation location

=Process=

Right now the vast majority of our items that are text descriptions in the UI (info about an object that is clicked on) are done from an InfoFactory, specifically Gui2InfoFactory. This will replace the behavior in those methods.

These items should be converted one at a time, so code reviews can be done in between each item, so that we know certain automation (macros, etc) are being appropriately produced. (This isn't to say we won't discover things later, but let's catch what we can by inspection).

=Success=

This includes:
# The HTML elements are externalized from Gui2InfoFactory
# The HTML is processed through FreeMarker before it is written into the UI
## Place the Freemarker processing into pcgen.io for now - probably into a new class.
## Please reuse your freemarker.template.Configuration.
## Consider reusing the freemarker.template.Template objects - if you don't, have a reasoning for why not.
# All of the getHTMLInfo methods that build HTML are modified and have the source HTML extraced to external files.
## Note: After those methods are complete, it can be evaluated what other methods it is valuable to extract as a "Phase 2".
## Note: Communication with the release team will be necessary to make sure these files are included in the release
# For purposes of this project the *Facade objects should not be used during export. If you look at how the wrappers in pcgen.output work, the UI Facades are not used, so pass in a "real" object to the output system when building the Map (or whatever style you use).
# You SHOULD be writing Freemarker macros, and it's okay to require importing of some macro files at the top of the HTML files.
## Formatting PI strings should not be done brute force every time, but SHOULD be done in Freemarker (not the code).
## Handling Yes/No for boolean objects should also be a macro handled in Freemarker.
## Other items like gathering Facts may also be useful to have macros. Don't go crazy, but make the info files easy to write as well. Yes, in certain points of view, this means the project has imprecise requirements. Get opinions and justify the decision.
# The Gui2InfoFactory knowledge is limited to a specific file name.
## For now, this file name can be hardcoded, and one file can exist for all of PCGen. It will be a follow-on project to make that file specific to a game mode and copied into the game mode directories. I would rather do that all at once across a number of files, so that the imposition on the system directory is in one PR.
## Share the actual export process and only have the file name determination be part of the Gui2InfoFactory.

=Implicit Design=

In addition to the macros pointed out above, there are a number of items that must implicitly be designed to be successful here. This includes (but may not be limited to):
# Determining how to communicate the SourceFormat preferences to the output. Note that today this is a SourceFormat object returned by Globals
## There should probably be multiple formatting macros in Freemarker that can be selected?
## This also needs consideration of how to keep this reasonable both short and long term, with the understanding that use of Globals strategically isn't a preferred practice.
# Dealing with prerequisites and Freemarker output.
## You should be aware Prerequisites in their current form are NOT strategic. Therefore, you should probably just leverage the existing HTML that they produce and figure out a way to get that to the Freemarker output without worrying about any clever redesign.
# How will you pull an internationalized string from the core code (assuming the HTML knows the internatinoalization key)?

=Documentation Required=
You will need to document at least:
# What is the active object is called in the HTML? It should be self-consistent across the files. Note that this does not mean it always has to be "active" or some such, it could be "race" and "template" for different object types. Both are self-consistent under this definition. Feedback from the wider team (especially data/output) is encouaged.
# Where are the files placed?
# What are the files called?
# What files need to be included in the release (if not obvious from the above)

=Other Notes=
You WILL run into items which are not currently exported in the Freemarker output system. We WILL need to talk about these.

=Pointers=
You should read pcgen.io.ExportHandler.exportCharacterUsingFreemarker to see what is done there since you will need a simpler version of that process.

Code

2018-03-26T21:19:13Z

Tom Parker: /* Open Sub Projects */

{| align="right"
| __TOC__
|}

=Introduction=
Welcome to the Wiki section for the Code Monkey team!

=Mission Statement=
'''TBA'''

=Resources=
* [[Developers Meeting]]
* [[Dev Meeting Log 20090429]]
* [[Dev Meeting Log 20090815]]
* [[Dev Meeting Log 20090911]]
* [[Dev Meeting Log 20091017]]
* [[Dev Meeting Log 20091114]]
* [[Dev Meeting Log 20120309]]
* [[Dev Meeting Log 20120429]]
* [[Dev Meeting Log 20121103]]
* [[Dev Meeting Log 20140627]]

==For New Starters==
* [[Joining the Code Team]] General steps to take to prepare yourself
* [[Code Monkey Introduction]]

==Developer Environment==
* [[Basic Developer Setup]]
* [[Git Setup]]
* [[Subversion Setup]]
* [[Merging]] with SVN
* [[Building PCGen]], also see [[Gradle]]
* [[Continuous Integration]]
* [[launch4j]]

==Coding==
* [[Coding Standards]]
* [[Deprecating a Token]]
* [[Explanation of the Code Base]]
* [[PCGen Code Exploration Almanac]]
* [[Graph Theory]]
* [[Logging in the Code Base]]
* [[Unit Testing]]
* [[Removing Libraries]]
* [[Unnecessary Code Detector]]

=Completed Sub Projects=
* [[Ability Object TODO List]]
* [[LST_Editor_Verification]]

=Open Sub Projects=
* [[UI HTML Extraction]]
* [[Internationalization]]
* [[CDOM]]
* [[UI Overhaul]]
* [[LST_Editor]]
* [[Major Code Projects]]
* [[Template Engine]]
* [[Enhanced Networking Features]]

=Active Team Members=

===[[Explanation of Teams#Silverback|Silverback]]===
* Connor Petty

===[[Explanation of Teams#Second|2nd]]===
* TBA

===[[Explanation of Teams#Chimp|Chimp]]===
* [[Tom Parker]]
* [[Stefan Radermacher]]

===[[Explanation of Teams#Gibbon|Gibbon]]===
* Jayme Cox

===[[Explanation of Teams#Tamarin|Tamarin]]===
* [[Andrew Wilson]]
* [[Martijn Verburg]]
* [[Connor Petty]]

===[[Explanation of Teams#Lemur|Lemur]]===
* [[Eddy Anthony]]
* Per Christian Henden
* Thomas Cooper
* [[Tod Milam]] - Mac Developer
* [[Kim Winz]]

===[[X-Terminator Mark II]]===
* [[Greg Bingleman]] - [http://www.codemonkeypublishing.com Code Monkey Publishing]

===Other===
* Brad Stiles, SVN advice
* Jason Buchanan, Advice on old code
* [[Jonas Karlson]], Advice on old code

=Passive Team Members=

* B. K. Oxley (Provides IntelliJ licenses)

=Inactive Team Members=

===[[Explanation_of_Teams#2nd|2nd]]===
* Devon Jones

===[[Explanation_of_Teams#Chimp|Chimp]]===
* Aaron Divinsky

===[[Explanation_of_Teams#Tamarin|Tamarin]]===
* Andriy Sen
* [[Kevin Fernandes]]
* [[Koen Van Daele]]
* [[Joe Frazier, Jr.]]
* Per Christian Henden

===[[Explanation_of_Teams#Lemur|Lemur]]===
* Bill Neumann
* Dan Parks
* [[Eduard Martinescu]]
* Eric Jarman
* Jasper Spaans
* Julio Esslinger Viegas
* Peter Barker
* [[Rick Ryker]]
* Thomas Clegg
* Tony Lavalle
* Hades Lucifer [7/20/10]
* Josh Johnston [7/20/10]
* MotorViper [7/20/10]
* [[Steven West]] [7/20/10]
* [[Alec Ross]] [7/20/10]
* [[Tir Gwaith]] [7/20/10]

===Other===
* [[Andargor]], Suggestions
* [[Bryan McRoberts]], Advice on old code
* Kurt Wimmer, Suggestions
* Walter Duncan, Held the old pcgen_autobuild user

Race Racial Trait Swapping

2018-03-09T03:22:53Z

Tom Parker: /* Considerations */

{| align="right"
| __TOC__
|}

=Background=

Note that the basics for [[Replacing Archetype Controls]] is available

=Challenge=

Race is granted 'default' traits as a standard. Paizo has introduced two systems of concern:
# "Race Package" where a race is granted specific non-default traits
# "Race Package" where a race if granted a choice between two or more additional choices
# Racial Trait that is chosen by the player actually removes 2 or more existing default traits

=Potential Solutions Challenge #1=

This seems straight-forward to me. The Race Package can simply have:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|DifferentFeature|PRIORITY=400
</pre>

The variable assignments are very similar to what is shown in [[Replacing Archetype Controls]].

=Potential Solutions Challenge #2=

This involves a choice, thus interacts with [[SELECTION - A CHOOSE Replacement]]. This section is deferred until some changes and clarifications are made on that page with respect to how instances will work (there are multiple methods of causing instances and that is not currently clear).

=Potential Solutions Challenge #3=

I'm not sure there are multiple possible solutions here. I think this is straightforward.

==Filler Object==

Let's assume we have a Racial Trait "Super" that replaces "Good" and "Lucky".

This presumes we have done:
<pre>
DYNAMICSCOPE:RACEFEATURE
</pre>

In VARIABLE file:
<pre>
LOCAL:RACE|RACEFEATURE=Feature_Lucky
LOCAL:RACE|RACEFEATURE=Feature_Good
</pre>

So on our race we had:
<pre>
MODIFY:Feature_Lucky|SET|Default_Feature_Lucky
MODIFY:Feature_Good|SET|Default_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|SuperFeature|PRIORITY=400
</pre>

Then we need to remove Default_L3_Feature_Lucky. I would probably recommend something like:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Lucky|SET|EmptyFeature|PRIORITY=400
</pre>

EmptyFeature would then be an object with nothing in it.

===Justification===

Keep in mind if we have a DYNAMIC of RACEFEATURE (which Default_Feature_Lucky is one implementation) then we need a default value for that object type. So it's already likely that we have an "EmptyFeature" set up to be the default value for a variable of the RACEFEATURE FORMAT.

==Array==

One possible alternative would be to have an ARRAY[RACEFEATURE] that contains all of the traits for a given level.

In VARIABLE file:
<pre>
LOCAL:RACE|ARRAY[RACEFEATURE]=Features
</pre>

So on our race we had:
<pre>
MODIFY:Features|ADD|Default_Feature_Lucky+Default_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Features|SET|SuperFeature|PRIORITY=400
</pre>

===Considerations===

In this case, we need to look at what it would be like for a class to replace only one of the features.

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Features|SET|value()-Default_Feature_Good+Default_Feature_Better|PRIORITY=400
</pre>

or alternatively:
<pre>
MODIFYOTHER:RACE|Elf|Features|ADD|Default_Feature_Better|PRIORITY=400
MODIFYOTHER:RACE|Elf|Features|REMOVE|Default_Feature_Good|PRIORITY=400
</pre>

Considerations:
# We have avoided adding an empty shell to the PC
# We are making (risky?) assumptions of what we are replacing
# We start to get into order of operations problems between features that touch different features (think of this more broadly in terms of Archetypes that can impact different items at the same level both being usable together), so it increases the cross-data interaction

In general, I think this is a bad idea due to the built-in assumptions.

Race Racial Trait Swapping

2018-03-09T03:22:08Z

Tom Parker: /* Array */

{| align="right"
| __TOC__
|}

=Background=

Note that the basics for [[Replacing Archetype Controls]] is available

=Challenge=

Race is granted 'default' traits as a standard. Paizo has introduced two systems of concern:
# "Race Package" where a race is granted specific non-default traits
# "Race Package" where a race if granted a choice between two or more additional choices
# Racial Trait that is chosen by the player actually removes 2 or more existing default traits

=Potential Solutions Challenge #1=

This seems straight-forward to me. The Race Package can simply have:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|DifferentFeature|PRIORITY=400
</pre>

The variable assignments are very similar to what is shown in [[Replacing Archetype Controls]].

=Potential Solutions Challenge #2=

This involves a choice, thus interacts with [[SELECTION - A CHOOSE Replacement]]. This section is deferred until some changes and clarifications are made on that page with respect to how instances will work (there are multiple methods of causing instances and that is not currently clear).

=Potential Solutions Challenge #3=

I'm not sure there are multiple possible solutions here. I think this is straightforward.

==Filler Object==

Let's assume we have a Racial Trait "Super" that replaces "Good" and "Lucky".

This presumes we have done:
<pre>
DYNAMICSCOPE:RACEFEATURE
</pre>

In VARIABLE file:
<pre>
LOCAL:RACE|RACEFEATURE=Feature_Lucky
LOCAL:RACE|RACEFEATURE=Feature_Good
</pre>

So on our race we had:
<pre>
MODIFY:Feature_Lucky|SET|Default_Feature_Lucky
MODIFY:Feature_Good|SET|Default_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|SuperFeature|PRIORITY=400
</pre>

Then we need to remove Default_L3_Feature_Lucky. I would probably recommend something like:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Lucky|SET|EmptyFeature|PRIORITY=400
</pre>

EmptyFeature would then be an object with nothing in it.

===Justification===

Keep in mind if we have a DYNAMIC of RACEFEATURE (which Default_Feature_Lucky is one implementation) then we need a default value for that object type. So it's already likely that we have an "EmptyFeature" set up to be the default value for a variable of the RACEFEATURE FORMAT.

==Array==

One possible alternative would be to have an ARRAY[RACEFEATURE] that contains all of the traits for a given level.

In VARIABLE file:
<pre>
LOCAL:RACE|ARRAY[RACEFEATURE]=Features
</pre>

So on our race we had:
<pre>
MODIFY:Features|ADD|Default_Feature_Lucky+Default_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Features|SET|SuperFeature|PRIORITY=400
</pre>

===Considerations===

In this case, we need to look at what it would be like for a class to replace only one of the class features at that level.

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Features|SET|value()-Default_Feature_Good+Default_Feature_Better|PRIORITY=400
</pre>

or alternatively:
<pre>
MODIFYOTHER:RACE|Elf|Features|ADD|Default_Feature_Better|PRIORITY=400
MODIFYOTHER:RACE|Elf|Features|REMOVE|Default_Feature_Good|PRIORITY=400
</pre>

Considerations:
# We have avoided adding an empty shell to the PC
# We are making (risky?) assumptions of what we are replacing
# We start to get into order of operations problems between features that touch different features (think of this more broadly in terms of Archetypes that can impact different items at the same level both being usable together), so it increases the cross-data interaction

In general, I think this is a bad idea due to the built-in assumptions.

Race Racial Trait Swapping

2018-03-09T03:19:25Z

Tom Parker: /* Filler Object */

{| align="right"
| __TOC__
|}

=Background=

Note that the basics for [[Replacing Archetype Controls]] is available

=Challenge=

Race is granted 'default' traits as a standard. Paizo has introduced two systems of concern:
# "Race Package" where a race is granted specific non-default traits
# "Race Package" where a race if granted a choice between two or more additional choices
# Racial Trait that is chosen by the player actually removes 2 or more existing default traits

=Potential Solutions Challenge #1=

This seems straight-forward to me. The Race Package can simply have:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|DifferentFeature|PRIORITY=400
</pre>

The variable assignments are very similar to what is shown in [[Replacing Archetype Controls]].

=Potential Solutions Challenge #2=

This involves a choice, thus interacts with [[SELECTION - A CHOOSE Replacement]]. This section is deferred until some changes and clarifications are made on that page with respect to how instances will work (there are multiple methods of causing instances and that is not currently clear).

=Potential Solutions Challenge #3=

I'm not sure there are multiple possible solutions here. I think this is straightforward.

==Filler Object==

Let's assume we have a Racial Trait "Super" that replaces "Good" and "Lucky".

This presumes we have done:
<pre>
DYNAMICSCOPE:RACEFEATURE
</pre>

In VARIABLE file:
<pre>
LOCAL:RACE|RACEFEATURE=Feature_Lucky
LOCAL:RACE|RACEFEATURE=Feature_Good
</pre>

So on our race we had:
<pre>
MODIFY:Feature_Lucky|SET|Default_Feature_Lucky
MODIFY:Feature_Good|SET|Default_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|SuperFeature|PRIORITY=400
</pre>

Then we need to remove Default_L3_Feature_Lucky. I would probably recommend something like:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Lucky|SET|EmptyFeature|PRIORITY=400
</pre>

EmptyFeature would then be an object with nothing in it.

===Justification===

Keep in mind if we have a DYNAMIC of RACEFEATURE (which Default_Feature_Lucky is one implementation) then we need a default value for that object type. So it's already likely that we have an "EmptyFeature" set up to be the default value for a variable of the RACEFEATURE FORMAT.

==Array==

One possible alternative would be to have an ARRAY[CLASSFEATURE] that contains all of the traits for a given level.

So on our class we had:
<pre>
MODIFY:Level_3_Features|ADD|Default_L3_Feature_Lucky+Default_L3_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|SET|SuperFeature|PRIORITY=400
</pre>

===Considerations===

In this case, we need to look at what it would be like for a class to replace only one of the class features at that level.

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|SET|value()-Default_L3_Feature_Good+Default_L3_Feature_Better|PRIORITY=400
</pre>

or alternatively:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|ADD|Default_L3_Feature_Better|PRIORITY=400
MODIFYOTHER:CLASS|Fighter|Level_3_Features|REMOVE|Default_L3_Feature_Good|PRIORITY=400
</pre>

Considerations:
# We have avoided adding an empty shell to the PC
# We are making (risky?) assumptions of what we are replacing
# We start to get into order of operations problems between Archetypes that both touch the same level, but different parts, so it increases the cross-data interaction

In general, I think this is a bad idea due to the built-in assumptions.

Race Racial Trait Swapping

2018-03-09T03:18:16Z

Tom Parker: /* Filler Object */

{| align="right"
| __TOC__
|}

=Background=

Note that the basics for [[Replacing Archetype Controls]] is available

=Challenge=

Race is granted 'default' traits as a standard. Paizo has introduced two systems of concern:
# "Race Package" where a race is granted specific non-default traits
# "Race Package" where a race if granted a choice between two or more additional choices
# Racial Trait that is chosen by the player actually removes 2 or more existing default traits

=Potential Solutions Challenge #1=

This seems straight-forward to me. The Race Package can simply have:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|DifferentFeature|PRIORITY=400
</pre>

The variable assignments are very similar to what is shown in [[Replacing Archetype Controls]].

=Potential Solutions Challenge #2=

This involves a choice, thus interacts with [[SELECTION - A CHOOSE Replacement]]. This section is deferred until some changes and clarifications are made on that page with respect to how instances will work (there are multiple methods of causing instances and that is not currently clear).

=Potential Solutions Challenge #3=

I'm not sure there are multiple possible solutions here. I think this is straightforward.

==Filler Object==

Let's assume we have a Racial Trait "Super" that replaces "Good" and "Lucky".

So on our race we had:
<pre>
MODIFY:Feature_Lucky|SET|Default_Feature_Lucky
MODIFY:Feature_Good|SET|Default_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|SuperFeature|PRIORITY=400
</pre>

Then we need to remove Default_L3_Feature_Lucky. I would probably recommend something like:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Lucky|SET|EmptyFeature|PRIORITY=400
</pre>

EmptyFeature would then be an object with nothing in it.

===Justification===

Keep in mind if we have a DYNAMIC of RACEFEATURE (which Default_Feature_Lucky is one implementation) then we need a default value for that object type. So it's already likely that we have an "EmptyFeature" set up to be the default value for a variable of the RACEFEATURE FORMAT.

==Array==

One possible alternative would be to have an ARRAY[CLASSFEATURE] that contains all of the traits for a given level.

So on our class we had:
<pre>
MODIFY:Level_3_Features|ADD|Default_L3_Feature_Lucky+Default_L3_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|SET|SuperFeature|PRIORITY=400
</pre>

===Considerations===

In this case, we need to look at what it would be like for a class to replace only one of the class features at that level.

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|SET|value()-Default_L3_Feature_Good+Default_L3_Feature_Better|PRIORITY=400
</pre>

or alternatively:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|ADD|Default_L3_Feature_Better|PRIORITY=400
MODIFYOTHER:CLASS|Fighter|Level_3_Features|REMOVE|Default_L3_Feature_Good|PRIORITY=400
</pre>

Considerations:
# We have avoided adding an empty shell to the PC
# We are making (risky?) assumptions of what we are replacing
# We start to get into order of operations problems between Archetypes that both touch the same level, but different parts, so it increases the cross-data interaction

In general, I think this is a bad idea due to the built-in assumptions.

Race Racial Trait Swapping

2018-03-09T03:15:56Z

Tom Parker:

{| align="right"
| __TOC__
|}

=Background=

Note that the basics for [[Replacing Archetype Controls]] is available

=Challenge=

Race is granted 'default' traits as a standard. Paizo has introduced two systems of concern:
# "Race Package" where a race is granted specific non-default traits
# "Race Package" where a race if granted a choice between two or more additional choices
# Racial Trait that is chosen by the player actually removes 2 or more existing default traits

=Potential Solutions Challenge #1=

This seems straight-forward to me. The Race Package can simply have:
<pre>
MODIFYOTHER:RACE|Elf|Feature_Good|SET|DifferentFeature|PRIORITY=400
</pre>

The variable assignments are very similar to what is shown in [[Replacing Archetype Controls]].

=Potential Solutions Challenge #2=

This involves a choice, thus interacts with [[SELECTION - A CHOOSE Replacement]]. This section is deferred until some changes and clarifications are made on that page with respect to how instances will work (there are multiple methods of causing instances and that is not currently clear).

=Potential Solutions Challenge #3=

I'm not sure there are multiple possible solutions here. I think this is straightforward.

==Filler Object==

Let's assume we have a Racial Trait "Super" that replaces "Good" and "Lucky".

So on our class we had:
<pre>
MODIFY:Level_3_Feature_Lucky|SET|Default_L3_Feature_Lucky
MODIFY:Level_3_Feature_Good|SET|Default_L3_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Feature_Good|SET|SuperFeature|PRIORITY=400
</pre>

Then we need to remove Default_L3_Feature_Lucky. I would probably recommend something like:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Feature_Lucky|SET|EmptyFeature|PRIORITY=400
</pre>

EmptyFeature would then be an object with nothing in it.

===Justification===

Keep in mind if we have a DYNAMIC of CLASSFEATURE (which Default_L3_Feature_Lucky is one implementation) then we need a default value for that object type. So it's already likely that we have an "EmptyFeature" set up to be the default value for a variable of the CLASSFEATURE FORMAT.

==Array==

One possible alternative would be to have an ARRAY[CLASSFEATURE] that contains all of the traits for a given level.

So on our class we had:
<pre>
MODIFY:Level_3_Features|ADD|Default_L3_Feature_Lucky+Default_L3_Feature_Good
</pre>

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|SET|SuperFeature|PRIORITY=400
</pre>

===Considerations===

In this case, we need to look at what it would be like for a class to replace only one of the class features at that level.

On our Trait we can do:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|SET|value()-Default_L3_Feature_Good+Default_L3_Feature_Better|PRIORITY=400
</pre>

or alternatively:
<pre>
MODIFYOTHER:CLASS|Fighter|Level_3_Features|ADD|Default_L3_Feature_Better|PRIORITY=400
MODIFYOTHER:CLASS|Fighter|Level_3_Features|REMOVE|Default_L3_Feature_Good|PRIORITY=400
</pre>

Considerations:
# We have avoided adding an empty shell to the PC
# We are making (risky?) assumptions of what we are replacing
# We start to get into order of operations problems between Archetypes that both touch the same level, but different parts, so it increases the cross-data interaction

In general, I think this is a bad idea due to the built-in assumptions.

Archetype Swapping Concerns

2018-03-09T02:54:28Z

Tom Parker:

{| align="right"
| __TOC__
|}

=Background=

Note that the basics for [[Replacing Archetype Controls]] is available

=Challenge=

Class Archetypes: in some instances a Class Feature is not replaced, but a portion of the feature is negated in order to gain another new feature.

==Example==

Rogue gains Sneak Attack (Class Feature), which has level dependent gains.

Every odd level, the rogue gains an additional +1d6 to the Sneak Attack Damage.

Broken down:
1 = 1d6
3 = 2d6
5 = 3d6

However, we encounter this:
1 = 1d6
3 = New Class Feature
5 = 2d6

===This section is not well understood how it relates to the example===

We need to manage several things:
# When a new bonus is applied (We cannot remove/negate a bonus until the correct level is reached)
# We cannot grant new features until the minimum level has been reached
# And in odd cases, when features are granted, there can be level dependent bonuses on those as well...

=Principles & Discussion=

I've articulated a number of different principles that will come into play here. These include, but are not limited to:
# Be direct. Don't set a variable to go do something somewhere else, rather, directly set the variable needed
# Keep things as aligned as possible to how the rule construct actually works.

This example is useful, because it clarifies what I mean by "direct", in that it is still the owning object making the modification, but that being super-literal about being direct being the actual *result* can have bad implications.

As a note, in the Example provided above, we should go back to how the rule is actually designed.

I believe in this case it's something like "At level 1, you are granted 1d6 Sneak Attack Damage". At Level 3 it's "An additional 1d6".

There are 2 possible interpretations of what to do at level 3:
* Think in Set Dice: Set the total directly to 2d6
* Add a Die: Add 1 additional die to the running total

Unfortunately, I'm not sure I understand all of the listed things that need to be managed (since I'm not sure they are all handled in the example, and generalizations about things I don't understand are dangerous... :) )... so it's probable that other examples need to be provided so that additional implementations can be provided.

=Potential Solutions=

Note: This presumes you are familiar with how items would be replaced, since it will work in a similar fashion to other Archetype replacements. This is only dealing with the incremental dice and how it would be managed to have a "middle one" replaced.

==Thinking in Set Dice==

One solution is to think fully in set values of dice:

In VARIABLE file:
<pre>
LOCAL:CLASS|DICE=SneakAttack
</pre>

On some class feature:
<pre>
DoAtLevel3 <> MODIFY:SneakAttack|SET|2d6|PRIORITY=400
</pre>

I think this has a critical flaw:
# If there needs to be an interrupt (as in the example above), then this will fail (or there will need to be some variable passing around what did or didn't happen earlier). In general, while this is "brute force" correct, it fails to have any flexibility and thus can't meet the requirements.

I happen to think this also fails to follow the principle summarized as "do directly what the book says". While direct, this is direct to the *result*, not to the *operation specified by the book* (the operation being add 1d6)

==Adding a Die==

One solution is to think in modifying dice:

In VARIABLE file:
<pre>
LOCAL:CLASS|DICE=SneakAttack
</pre>

<pre>
DoAtLevel3 <> MODIFY:SneakAttack|ADDDIE|1|PRIORITY=300
</pre>

Some considerations:
# This solves the issue in the first example, in that missing a class feature at a given level will not cause the total to be incorrect.
# This is flexible to different sizes of dice, so if for some reason there is a method of increasing the die size of Sneak Attack, then this will still succeed.

=Other Commentary=

==Why doesn't Dice support ADD?==

Based on the literal rules of adding 1d6, you might ask why you can't do this:
<pre>
DoAtLevel3 <> MODIFY:SneakAttack|ADD|1d6|PRIORITY=300
</pre>

If ADD was supported for DICE, this would solve the issue in the first example, in that missing a class feature at a given level will not cause the total to be incorrect.

However, this introduces a major problem.

Remember that the DICE format is equal to:
<pre>
XdY+Z
</pre>

If we allow ADD, we end up with a problem. What if we SET it to 1d8 and then try to add 1d6? We now have two different values for Y, so it can't be processed with the format DICE.

===So wait, what about my sword damage?!?===

I need 1d8+1d6(fire). How do I get that?

The answer is the damage on a sword will be FORMAT of ARRAY[DICE].

6.08 Documentation

2018-03-08T18:26:54Z

Tom Parker: /* New Documentation Repository for 6.08 */

==New Documentation Repository for 6.08==

[[Setting up the new Formula System]] - How to set up and use variables (MODIFY, MODIFYOTHER, DYMAMIC, GRANT, FUNCTION, VALUE)

[[Lookup & TABLE 6.08]] - Lookup is used with Table

[[INFO & INFOVARS 6.08]] - Dynamic Text Display (Replaces DESC, BENEFIT, ASPECT, QUALITY, SPROP)

[[DICE in Formula 6.08]] - New System for handling Dice Formats

[[CODE CONTROLS]] - Disable specific tags and replaces it with a FORMULA Version (For output sheets integration)

[[Replacing Archetype Controls]] - Some initial thoughts on how to use the formula system to make Archetypes easier

[[OUTPUT for 6.08]] - Formats for output into Freemarker

Replacing Archetype Controls

2018-03-08T18:26:51Z

Tom Parker: Created page with "{| align="right" | __TOC__ |} =Current State= Archetype prerequisites are fairly complex. Currently, there are a tremendous number of FACTS (literally thousands) used..."

{| align="right"
| __TOC__
|}

=Current State=

Archetype prerequisites are fairly complex. Currently, there are a tremendous number of FACTS (literally thousands) used to control what Archetype can be taken. More items are then used to determine how to apply each item. Much of this depends on the Default object and trailing PRExxx. Following the logic is ... difficult, and the performance is probably not great, due to lots and lots of trailing PRExxx.

=Using the new formula system=

==Limiting Selection==

First of all, we define a new type of object called LOCK:

In a DATACONTROL file:
<pre>
DYNAMICSCOPE:LOCK
</pre>

We then define each type of lock:

In a DYNAMIC File:
<pre>
LOCK:Fighter_3_SomethingOrOther
</pre>

Now we want a place to store all the locks we have encountered on the PC:

In a VARIABLE file:
<pre>
GLOBAL:ARRAY[LOCK]=AllLocks
</pre>

Now we recognize that we want 2 things to happen:
# If an Archetype needs a Lock that someone else already has it should not be allowed
# An Archetype should grant all of the locks it needs (to "claim" them)

Note: At this point, we need to suspend disbelief for a moment, since it's not quite defined if Archetypes will be Ability objects or if they are better served as something else. For the rest of this explanation, it assumes there is a type called ARCHETYPE to avoid too much complexity in the example.

There are 2 things that need the locks, and in software development, it tends to be useful to type things once and use them in two places, to avoid things becoming inconsistent. In this case, we can use a FACTSET to do that:

We define the FACTSET:
<pre>
FACTSETDEF:ARCHETYPE|Locks <> DATAFORMAT:LOCK
</pre>

Then we put the locks in a FACTSET on the Archetype:
<pre>
MyArchetype <> FACTSET:Locks|Fighter_3_SomethingOrOther
</pre>

Now we need to use that FACTSET. To avoid allowing the Archetype to be used if any of its Locks are already claimed we use this:
<pre>
ALLOW:isEmpty(overlap(AllLocks,getFactSet("ARCHETYPE", this(), "Locks")))
</pre>

This is very much new relative to other formulas so let's explain this in a bit of detail.

This uses the "this()" function. We could have hardcoded the FACTSET to look up the Archetype itself, e.g.:
<pre>
getFactSet("ARCHETYPE", "MyArchetype", "Locks")
</pre>
...however that encounters a problem if we .COPY MyArchetype... so we don't want that hardcoding. We use the this() function to allow the reference to automatically update to the object on which the item is applied.

Second, it performs the function overlap:
<pre>
overlap(X,Y)
</pre>

This hasn't been seen in previous examples.

This will take two arrays and return a subset of items that are in both arrays.

So AllLocks is ARRAY[LOCK], and the getFactSet call will also return an ARRAY[LOCK]. Any LOCK in both arrays (which indicates a conflict which should prohibit application of the Archetype) will be in the returned array.

We then ensure that array is empty:
<pre>
isEmpty(...)
</pre>

If it's empty, taking the Archetype is allowed.

Lastly, we need to have the Archetype grant its locks. We can do this without having a token on each Archetype by using the Global Modifier file:
<pre>
MODIFYOTHER:ARCHETYPE|ALL|AllLocks|ADD|getFactSet(target(),"Locks")
</pre>

This uses the "target()" function. We want to avoid placing this MODIFYOTHER on every Archetype - ANNOYING. We want to do that once, so we put it in the Global Modifier file and it will apply to all Archetypes. Note just like the logic on "this()" earlier, we have a function called "target()" (which is where the MODIFYOTHER is applied) which is generic enough that we can write this token ONCE and have it apply to every Archetype.

So here is what we have:
# We've reduced each Archetype to having only two tokens to work on overlap (the FACTSET and the APPLY limitation)
# We have used a separate object type LOCK in order to have the FACTSET be typo-safe.
## Meaning it's not just general String in the FACTSET, so you have to consciously put the FACT in one file and then use it in other LST files

==Overriding the Feature==

Now each Archetype needs to override a class feature. To do that, we simply want to treat each class feature as a variable, so it can be easily overridden using the MODIFY* system:

In VARIABLE file:
<pre>
LOCAL:CLASS|ARCHETYPE=Level_3_Feature
</pre>

In CLASS LST file:
<pre>
CLASS:Fighter <> MODIFY:Level_3_Feature|SET|Default_L3_Feature_Fighter
CLASS:Fighter <> GRANTVAR:Level_3_Feature
</pre>

Note that this puts an Archetype into a Variable and then "grants" the contents of that variable to the PC. If we want to change what is granted, we simply need to change the contents of that variable:

In ARCHETYPE LST file:
<pre>
MyArchetype <> MODIFYOTHER:CLASS|Fighter|Level_3_Feature|SET|SomeCustomFeature|PRIORITY=400
</pre>

Note a few advantages here as well:
# This is all DIRECTLY in the objects either owning (CLASS) or modifying (ARCHETYPE) the situation. No setting variables to go trigger a PRExxx that is somewhere over on a Default object. If you can't write something directly in the new system, SPEAK UP. The goal is for it to be very straightforward.
# This is now all in "forward logic" - there are no trailing PRExxx here that can "turn things on or off" (and have a performance impact because they have to constantly be tested). Either the base feature is present, or it's overridden - reconsideration is only necessary at the moment a new Archetype is applied, and then we "know" with certainty whether the base or custom feature is present.

==Gaps as of March 8 2018==

In short, this is close to being doable, but not doable with the current master branch. It is hoped that this is doable by the time we reach 6.8. What specifically is needed:
# Currently isEmpty(X) is implemented but has not yet been merged with the master branch. It is currently flowing through from the Formula library, and should be merged well before the 6.8 release.
# Currently GRANTVAR exists in a local branch, but has not yet been submitted for a PR due to complexity and lots of other competing PRs.
# Currently this() and target() are in a pending PR that has received comments and is not yet merged.
# Currently overlap(X,Y) is not implemented. It is fairly trivial, but will be done in the formula system and take a library update integration to occur. It should occur by the 6.8 release.
# We need to have a discussion over what ARCHETYPE is (and whether it must be an Ability). GRANTVAR and Abilities are likely to not get along well (due to Category/Nature/etc). So we need to decide if these HAVE to be Ability objects
## If so, we will need GRANTVAR to support Abilities, or some other alternative
### This may require COMPOUND objects to be merged into the master branch (a local branch, not submitted for a PR yet)
## If not, we need to understand what features are required, since it will likely be a DYNAMIC
### This will require Interface Tokens to be merged into the master branch (a local branch, not submitted for a PR yet), so that DYNAMIC objects can contain MODIFY* tokens.
### This may also require additional features to be implemented on DYNAMIC objects

Tags and Tokens

2018-03-05T02:26:49Z

Tom Parker:

{| align="right"
| __TOC__
|}

==Overview==

Data that defines the rule constructs of a Class, Race, Language, and other characteristics and attributes of a PlayerCharacter are stored in LST files. These LST files are parsed to import the information from those LST files into objects within the PCGen program.

For purposes of the architecture descriptions here on the Wiki, the individual entries in the data persistence files will be referred to as "Tags". To store information, these tags separate the "key" (used to indicate the tag name) and the "value" (the information the tag is conveying to PCGen) by a colon. A simple example can be taken from the Common Language in the Language LST file from SRD 3.0:
TYPE:Spoken
This indicates the "TYPE" tag is indicating a value of "Spoken".

For purposes of this document, the code that parses the value of a tag and translates the tag syntax into the internal data structure of PCGen is called a Token.

Occasionally, you fill find these the terms "tag" and "token" used differently; which name is used will likely depend on the perspective of the speaker. The Wiki makes effort to use the terms Tag and Token in the context they are defined in this section. This is a decision for clarity, and is not a judgement of the "best" method for referring to the items in the data persistence format or the code.

Types of tokens (meaning the Interfaces that describe the behavior for each type of token) are located in pcgen.rules.persistence.token.

Tokens that implement these interfaces, which are effectively extensions to the [[Rules Persistence System]], are found in the plugin.lsttokens package.

==Types of Tokens==

===CDOMToken===

A CDOMToken is a PCGen 5.16 token that has a parse() method for processing the value of a tag and a getTokenName() method to identify the name of the tag the Token can process. Note that a CDOMToken does not require unparse(), as compatibility tokens from previous revisions would not implement that behavior.

===CDOMPrimaryToken===

A CDOMPrimaryToken is a CDOMToken that is active (not deprecated) in the current version of PCGen, and thus implements the unparse() method.

===CDOMSubToken===

A CDOMSubToken is a CDOMToken that is designed to operate for [[Rules_Persistence_System#Subtokens|Subtokens]]

===CDOMSecondaryToken===

A CDOMSecondaryToken is a CDOMSubToken that is active (not deprecated) in the current version of PCGen, and thus implements the unparse() method.

===CDOMCompatibilityToken===

A CDOMCompatibilityToken is a CDOMToken that implements a tag syntax from a previous version of PCGen, see [[Rules_Persistence_System#Token Compatibility|Token Compatibility]]. Once deprecated, a CDOMPrimaryToken would become a CDOMCompatibilityToken.

===CDOMCompatibilitySubToken===

A CDOMCompatibilitySubToken is a CDOMSubToken that implements a tag syntax from a previous version of PCGen, see [[Rules_Persistence_System#Token Compatibility|Token Compatibility]]. Once deprecated, a CDOMSecondaryToken would become a CDOMCompatibilitySubToken.

===ClassWrappedToken===

A ClassWrappedToken is a specialized CDOMToken designed to provide compatibility for using CLASS tokens on the level 1 CLASSLEVEL line, see [[Rules_Persistence_System#Class Wrapped Token|Weaknesses of Class Wrapped Tokens]].

===PreCompatibilityToken===

A PreCompatibilityToken is a token for wrapping the PCGen 5.14-type Prerequisite tokens into 5.16-type tokens, see [[Rules_Persistence_System#Prerequisite Tags|Weaknesses of Prerequisite Tokens]].

===DeferredToken===

===PostValidationToken===

===PostDeferredToken===

===PrimitiveToken===

===QualifierToken===

==Note about Token Behavior==

A Note of Caution on Tags/Tokens: Different tags have different behavior with respect to overwriting or appending values. Some tags may only allow a single value (e.g. RACETYPE from Race LST files), and subsequent references to that tag in the same object will overwrite the original value. Other tags are effectively lists (e.g. the Global TYPE tag), and subsequent calls to that tag will append values to the list. The behavior is noted within the tag documentation distributed with PCGen.

Formula Systems

2018-03-05T02:25:30Z

Tom Parker: /* New Capabilities */

{| align="right"
| __TOC__
|}

=JEP Formula System=

==What it is==

The formula system is embedded within the "core" of PCGen to do mathematical calculations. It is used internally as well as exposed in a limited fashion to the data developers through tokens. Specifically, both data defined variables and BONUS values depend on the formula calculations performed by this subsystem.

==Licensing==

The licensing for JEP changed and we have targeted it for replacement. When we originally integrated JEP, it was licensed under a dual commercial/GPL license. We received (and still operate under) a special exception to use JEP with our code. This has the side effect of limiting the use of our code outside of our distribution. Subsequent to our initial use of JEP, the GPL option was dropped, and JEP is now a commercial product. This means we no longer have updates available to us, and we are using a stagnant library.

==Performance==

We want to improve the performance of formula calculation. Today, each time a formula is processed, we re-parse the formula (which redoes all of the validation and other checks). This is CPU intensive for complex formulas, and should be something we can cache and re-use. We therefore want a system where we can parse the formula early in the process and store the parsed formula, using that "binary" version for evaluation.

We want to improve performance around BONUSes. The major performance bottleneck we now have is around the loop of "Variables can be modified by BONUSes, which can be conditional upon Prerequisites, which can use variables". This loop is currently "lookback" in that things are calculated as necessary and a large resolution loop is required to ensure the system reaches stability. We want to shortcut this when possible to reduce looping and overall calculation of values that do not change.

==Current Status==

We are working to remove JEP from PCGen. This is a result of
# JEP is now a closed source library, so we cannot continue to benefit from future releases/bug fixes
# JEP is a license exception in our code, which restricts our license flexibility
# JEP has more features than we need (some of which get in our way)

A new set of code (imported as the PCGen-Formula library, stored in a separate GitHub project called PCGen-Formula) is the replacement. It addressed many of the requirements we are working to get, including local variables and validation at LST load (rather than runtime). By also maintaining an awareness of direct dependencies, it can perform a dramatically smaller number of calculations when any value is changed.

==Format==

The Format of a variable is the java "Class" it contains. In JEP, all variables are numbers.

==Parsing==

Major characteristics of this system:
* Formulas have functions (delimited by parenthesis)
* Formulas have both built-in terms as well as user defined variables (DEFINE: token)
* All user defined variables are global in scope
* All user defined variables are provided a starting value on definition
* User defined variables are assumed to be zero if no DEFINE is ever encountered
* User defined variables may be defined in more than one location
* Some terms may be local in scope (e.g. spells have unique terms), but these are fragile and not thread safe.
* Diagnostic tools in the UI allow presentation of the current value of a variable (or really any formula)
* Formulas are NOT parsed for significant levels of validity at data load. They are given basic checks (to ensure parenthesis match, for example), but a full validity check is not possible.

==Design==

Currently, the use of JEP requires a multi-pass resolution. The formula is parsed, then it is processed/queried in order to determine any variables or terms it contains, and then those values are loaded into the formula and the formula is resolved. (see VariableProcessor.processJepFormula() )

This (JEP) method of resolution comes with some challenges and risks.

The parsed version of a JEP formula (a PJEP) ends up with a lot of complexity. It is a controller of sorts, rather than just a framework for a formula.

Also, the parsed version of a JEP formula has a set of awareness. The variable/term values are loaded into it, meaning a contract is placed on the programmer to "clean that up" so those values do not pollute other, later calculations. (This is generally not an issue based on how it is otherwise used, but may not be initially clear to an uninformed reader)

=How we use JEP=

==Term==

For purposes of this document, a term is a "built in" value that can be used in a formula. These are a subset of the items currently documented in the "Pre-defined Variables" or "System-Defined Variables" section of the docs (depending on how you get to that section)

An example of this include (but are certainly not limited to) BAB, BASECR, CASTERLEVEL, CL, SR, and TL.

Note that some terms can possess context. For example, CASTERLEVEL is valid only in SPELL objects, as it requires the "context" of the castable spell (which implicitly includes the class level, DC, # of times usable, etc... it is more than the Spell object defined in the LST file)

Some built-in variables look to the user as if they are a function, and are often treated by users as functions even though they are terms. These include square brackets in the pre-defined variable name. (e.g. COUNT[SKILLS]).

==Variable==

For purposes of this document, a variable is a data-defined value. Using today's data, this means it was defined by the DEFINE: token being encountered in the data. Obviously, due to the nature of formulas and ambiguity, there is limitation on variable names in that they must not conflict with a pre-defined term.

===Global Variable===

Global variables are variables that exist across the entire set of data. A global variable can be defined in one object and used in another. This is currently the case for all variables in PCGen, as they are all created with the DEFINE: token.

==Paren Function==

A paren function is a function that uses parenthesis () to contain the arguments to the function. An example of this is var("CL=Fighter").

"var" is the function, "CL=Fighter" is the (one) argument to the function.

Called "paren functions" to avoid confusion with the "terms" that contain brackets.

==Ambiguity over a value==

We want to avoid some situations of ambiguity. The current system for variables takes a "largest wins" argument when multiple definitions are encountered. This leads to potential confusion (and potentially debate over whether such an implicit decision should be allowed). It has also led to the adoption of a "data standard" that all values should be defined at zero and modifications provided as a BONUS:VAR|... Our redesign looks to eliminate the confusion over multiple conflicting definitions by providing a global definition characteristic and preventing otherwise identical variable definitions from having different starting values.

=The new Formula parser=

==Design==

The new formula parser is a JavaCC (well, technically, JJTree) defined syntax. It is held separately in the PCGen-Formula library, which is one of our two [[PCGen Libraries]]

==Removed Characteristics==

===Boolean-aware calculation===

Boolean and Numeric values will be calculated in their own domain. It will NOT be assumed that TRUE is 1 and FALSE is 0 as some other formula systems assume. A Boolean value here will be a Boolean and only usable where a Boolean is legal.

Therefore, the common operations that are Boolean operations such as AND (&&) and OR (||) will only operate on Boolean values. If a Function requires a Boolean value (such as the first argument to an IF function), then it must validate the semantics of the subformula.

This was based on team feedback as the formula system was originally designed.

==Shared Characteristics==

Major characteristics of this system that are similar to the existing implementation:
# Formula Functions are stored as plugins, so that new functions would not necessarily force a full rebuild of PCGen.

==Changed Characteristics==

Major characteristics of this system that are major modifications of the existing implementation:
# Formats other than numbers are supported.
# Starting value for a variable is based on the variable format (e.g. zero for numbers), so multiple defines do not "compete" [new: Define does not have a definition per DEFINE: token, but rather a DEFAULTVARIBLEVALUE once per game mode for a variable format (numbers being one possible variable format). This is consistent with and thus formalizes the data best practice of defining variables to zero].
# Terms are eliminated. They should be replcaed by functions or user variables.
# BONUS:VAR is eliminated and replaced with Modifiers (see below) [and in general ALL BONUS tokens will eventually disappear].
# The new system will not have any terms. All items will be a function or a user defined variable. This will dramatically simplify dependency analysis.
# The system does "integer aware" arithmetic, meaning 2+3=5 (not 5.0 or 4.999999999 or 5.000000001 as we might have today).

==New Capabilities==
Major characteristics of this system that are new capabilities:
# Diagnostic tools in the UI should provide the ability to expose the entire calculation, from default value through all modifications (and source of those modifications).
## It is important to recognize that the data practice of indirection (e.g. one object awards a Template called FACE_20 that sets the FACE:) is STRONGLY discouraged in the new system. That indirection makes it A LOT harder to figure out what modified a variable - something that the new system can make VERY clear, but that the data can hide and make difficult.
# Formulas are validated at data load to ensure good syntax, and that any functions that are used exist (and are valid) and variables are nominally defined somewhere in the data.
## There are a few rare excpetions to full validation, because they really occur at runtime... so just like we can't automatically detect that a divide by zero will occur, we can't detect that a Table provided to a lookup() function will actually have the necessary values and columns for the incoming variables.
# Dependencies are tracked and any circular logic will be identified as such during calculation (it is expected that this is not possible to catch at LST load due to formulas on impossible data combinations leading to false positives). This is done in the specific Solver.
# Local Variables (see below)

===Local Variables===

Local Variables are variables that exist in only a portion of the data. They possess context. They can only be used within that context. For example, a local variable defined on a piece of Equipment could only be used within that piece of equipment. Since Equipment can "own" Equipment Modifiers, EqMods could also modify or evaluate the local variable. However, an attempt to interpret the value of the local variable in a Spell (for example) should produce an error, since there is no known context of Equipment within a spell.

Given that they have (per-item) context to a specific instance (such as "Longsword +1" and "Shortbow +2"), local variables have independent values on each instance (so MyPlusValue on the Longsword could be 1, and MyPlusValue on the Shortbow could be 2).

Like global variables, local variables are on a per-character basis (so local variables are per-character, per-item). A Longsword on one character has a value N and a Longsword on a second character has a value M. N may or may not be equal to M (there is no linkage - it is based on whether the two Longswords have identical EqMods).

==Modifiers==

The MODIFY and MODIFYOTHER tokens replace BONUS:VAR. These provide a significantly enhanced experience from using BONUS, specifically:
# Specific values calculated by the formula system
# No implicit stacking rules. All stacking is explicit by the data owner
# No implicit replacement rules. All replacement is explicit by the data owner
# Dependency will be explicit and calculation will be carefully managed. (A graph of dependencies between variables is maintained by a SolverManager)

==Single Pass Evaluation==

Unlike JEP where a pass is made to determine the variables and then load those into the object, the new formula system is capable of doing an evaluation in a single pass.

Having a single-pass resolution by a visitor to the tree, with the context passed in as a parameter, has a number of functional advantages:
* This effectively makes a formula "immutable" in the sense that the context is passed in during resolution and thus can be evaluated in multiple threads without causing issues. Note this is a necessary consideration if we want to reuse formulas that are in the data multiple times, as the UI demonstrably triggers evaluation of items from multiple threads. The visitors are also "immutable" (they are in fact, fully stateless), and thus reusable without being concerned about thread safety.
* This keeps the formula as light-weight as possible (The tree structure is still a bit expensive, but better than a PJEP).
* The context can be set based on what is passed in, meaning evaluation locality is driven by the caller (knowing the locality) rather than forcing the formula to evaluate some string to figure out where it is.
* The entire concept of a PJEP pool goes away (in exchange we effectively have a context, but those are reusable so we don't ever have the lock/free necessity of PJEP). (For specificity, those solving contexts are called "DependencyManager", "EvaluationManager", and "FormulaSemantics", and can currently be found in the PCGen-formula library.
* We actually prohibit any funky hoop-jumping. By only providing the context in the sense of a variable library, we can accurately handle things like spell-localized items (e.g. today's CASTERLEVEL term) without having a temporarily set global item.

=Further Reading=

For more detailed information on setting up the new formula system, see [[Setting up the new Formula System]]

PCGen Libraries

2018-03-05T02:21:25Z

Tom Parker: Created page with " =On Why they exist= There are two major reasons the new formlua system was put in a library: # None of it needs to be PCGen-specific, and the separate library forces that mo..."

=On Why they exist=

There are two major reasons the new formlua system was put in a library:
# None of it needs to be PCGen-specific, and the separate library forces that more gereric design thinking
# Eliminate the complexity of having JJTree/JavaCC within the primary code base. The jjtree/javacc calls required to build .java files make a more complex build cycle (the challenge being that you have to do a build, then actually select the project and cause it to refresh in order for it to correctly compile the java files to the current version). The number of times we used to require a form of refresh to the project after a JavaCC compile occurred was confusing a large portion of our population of code/data folks that build their own version of PCGen. We therefore extracted that complexity out so that only the code team working on the formula system will experience any of that.

Note that this had the effect of breaking a reasonable portion of the pcgen.base.* also into a separate (and different) sub-project since they are shared dependencies of the core and the formula parser. Since these items are reasonably stable (most have had no more than cosmetic changes in years), it is also a reasonable separation.

=On the Library Contents=

Early PCGen used almost fully reference semantic lists for storing information. This produced a number of really bad headaches, because folks felt the lists could be modified at any time and the calculations could come out correct. This was patently false, but it didn't prevent it from happening widely across the code base.

Much of the work in 5.8/5.10/5.12 was a very "hard" change to a value-semantic infrastructure. This means any list returned by CDOMObject and its descendents is simply a snapshot of the values at that time - the list was copied and ownership of the list was transferred to the caller to do with it whatever they want.

The reason I bring this up is I frequently get comments on the contents of PCGen-base and "you should go use Google Collections" or whatever. I appreciate the comments, but (a) maintaining these items is not where I'm spending my time (b) it's not where are problems are and (c) the semantics of those classes are almost universally reference semantics, so they are very different to use in an object-oriented system that expects to maintain appropriate behavior and isolation (and difficult to use less experienced developers who don't actually understand semantic differences).

In short, when our system is smaller, and we have the legacy systems removed so that we have a code base in the hundreds of classes (so folks can understand it much easier), then I think that is a viable and valuable discussion to have about the code base. Until then, I think the semantics of PCGen-base are required.

Index to Architecture Documents

2018-03-05T02:21:22Z

Tom Parker:

{| align="right"
| __TOC__
|}

This is effectively a "shorthand" list of documents (and a tiny bit of context around the links) for "A Day in the Life of PCGen"

=Resources=

Before reading about the details of the PCGen architecture, it is likely helpful to understand our key [[Design Concepts for PCGen]] and [[Architecture Document Conventions]]

=Building=

# You should understand we have [[PCGen Libraries]]

=Loading=

# The [[Startup System]] initializes PCGen and loads the initial user interface
# Files from Disk are loaded by the [[Rules Persistence System]]
# These files use [[CDOM References Concept Document|CDOM References]] to resolve order of operations conflicts during load and [[Referring to Groups in LST Data|Refer to Groups in LST Data]] to identify multiple possible objects (for say, a choice)
# The [[Load Commit Subsystem]] is used to commit information into the [[Rules Data Store]]
# Post Processing is completed as described in the [[Full Load Order Detail]]

=Using Data=

# When retrieving items from the Rules Data Store, it's important to [[Identifying Objects|Identify the right Object]]
# You will also need to know about [[Accessing Information]]
# And strategically, it's good to know how we are [[Becoming Data Driven]]

=Processing a PC=

# After [[Adding items to the PC]], we begin [[Calculating Items on the PC]]
# This calculation heavily depends on the [[Formula Systems]]
# Some of the items on a PC can be conditional, so it's valuable to understand [[Prerequisites and Requirements]]

Formula Systems

2018-03-05T02:21:17Z

Tom Parker: /* The new Formula parser */

{| align="right"
| __TOC__
|}

=JEP Formula System=

==What it is==

The formula system is embedded within the "core" of PCGen to do mathematical calculations. It is used internally as well as exposed in a limited fashion to the data developers through tokens. Specifically, both data defined variables and BONUS values depend on the formula calculations performed by this subsystem.

==Licensing==

The licensing for JEP changed and we have targeted it for replacement. When we originally integrated JEP, it was licensed under a dual commercial/GPL license. We received (and still operate under) a special exception to use JEP with our code. This has the side effect of limiting the use of our code outside of our distribution. Subsequent to our initial use of JEP, the GPL option was dropped, and JEP is now a commercial product. This means we no longer have updates available to us, and we are using a stagnant library.

==Performance==

We want to improve the performance of formula calculation. Today, each time a formula is processed, we re-parse the formula (which redoes all of the validation and other checks). This is CPU intensive for complex formulas, and should be something we can cache and re-use. We therefore want a system where we can parse the formula early in the process and store the parsed formula, using that "binary" version for evaluation.

We want to improve performance around BONUSes. The major performance bottleneck we now have is around the loop of "Variables can be modified by BONUSes, which can be conditional upon Prerequisites, which can use variables". This loop is currently "lookback" in that things are calculated as necessary and a large resolution loop is required to ensure the system reaches stability. We want to shortcut this when possible to reduce looping and overall calculation of values that do not change.

==Current Status==

We are working to remove JEP from PCGen. This is a result of
# JEP is now a closed source library, so we cannot continue to benefit from future releases/bug fixes
# JEP is a license exception in our code, which restricts our license flexibility
# JEP has more features than we need (some of which get in our way)

A new set of code (imported as the PCGen-Formula library, stored in a separate GitHub project called PCGen-Formula) is the replacement. It addressed many of the requirements we are working to get, including local variables and validation at LST load (rather than runtime). By also maintaining an awareness of direct dependencies, it can perform a dramatically smaller number of calculations when any value is changed.

==Format==

The Format of a variable is the java "Class" it contains. In JEP, all variables are numbers.

==Parsing==

Major characteristics of this system:
* Formulas have functions (delimited by parenthesis)
* Formulas have both built-in terms as well as user defined variables (DEFINE: token)
* All user defined variables are global in scope
* All user defined variables are provided a starting value on definition
* User defined variables are assumed to be zero if no DEFINE is ever encountered
* User defined variables may be defined in more than one location
* Some terms may be local in scope (e.g. spells have unique terms), but these are fragile and not thread safe.
* Diagnostic tools in the UI allow presentation of the current value of a variable (or really any formula)
* Formulas are NOT parsed for significant levels of validity at data load. They are given basic checks (to ensure parenthesis match, for example), but a full validity check is not possible.

==Design==

Currently, the use of JEP requires a multi-pass resolution. The formula is parsed, then it is processed/queried in order to determine any variables or terms it contains, and then those values are loaded into the formula and the formula is resolved. (see VariableProcessor.processJepFormula() )

This (JEP) method of resolution comes with some challenges and risks.

The parsed version of a JEP formula (a PJEP) ends up with a lot of complexity. It is a controller of sorts, rather than just a framework for a formula.

Also, the parsed version of a JEP formula has a set of awareness. The variable/term values are loaded into it, meaning a contract is placed on the programmer to "clean that up" so those values do not pollute other, later calculations. (This is generally not an issue based on how it is otherwise used, but may not be initially clear to an uninformed reader)

=How we use JEP=

==Term==

For purposes of this document, a term is a "built in" value that can be used in a formula. These are a subset of the items currently documented in the "Pre-defined Variables" or "System-Defined Variables" section of the docs (depending on how you get to that section)

An example of this include (but are certainly not limited to) BAB, BASECR, CASTERLEVEL, CL, SR, and TL.

Note that some terms can possess context. For example, CASTERLEVEL is valid only in SPELL objects, as it requires the "context" of the castable spell (which implicitly includes the class level, DC, # of times usable, etc... it is more than the Spell object defined in the LST file)

Some built-in variables look to the user as if they are a function, and are often treated by users as functions even though they are terms. These include square brackets in the pre-defined variable name. (e.g. COUNT[SKILLS]).

==Variable==

For purposes of this document, a variable is a data-defined value. Using today's data, this means it was defined by the DEFINE: token being encountered in the data. Obviously, due to the nature of formulas and ambiguity, there is limitation on variable names in that they must not conflict with a pre-defined term.

===Global Variable===

Global variables are variables that exist across the entire set of data. A global variable can be defined in one object and used in another. This is currently the case for all variables in PCGen, as they are all created with the DEFINE: token.

==Paren Function==

A paren function is a function that uses parenthesis () to contain the arguments to the function. An example of this is var("CL=Fighter").

"var" is the function, "CL=Fighter" is the (one) argument to the function.

Called "paren functions" to avoid confusion with the "terms" that contain brackets.

==Ambiguity over a value==

We want to avoid some situations of ambiguity. The current system for variables takes a "largest wins" argument when multiple definitions are encountered. This leads to potential confusion (and potentially debate over whether such an implicit decision should be allowed). It has also led to the adoption of a "data standard" that all values should be defined at zero and modifications provided as a BONUS:VAR|... Our redesign looks to eliminate the confusion over multiple conflicting definitions by providing a global definition characteristic and preventing otherwise identical variable definitions from having different starting values.

=The new Formula parser=

==Design==

The new formula parser is a JavaCC (well, technically, JJTree) defined syntax. It is held separately in the PCGen-Formula library, which is one of our two [[PCGen Libraries]]

==Removed Characteristics==

===Boolean-aware calculation===

Boolean and Numeric values will be calculated in their own domain. It will NOT be assumed that TRUE is 1 and FALSE is 0 as some other formula systems assume. A Boolean value here will be a Boolean and only usable where a Boolean is legal.

Therefore, the common operations that are Boolean operations such as AND (&&) and OR (||) will only operate on Boolean values. If a Function requires a Boolean value (such as the first argument to an IF function), then it must validate the semantics of the subformula.

This was based on team feedback as the formula system was originally designed.

==Shared Characteristics==

Major characteristics of this system that are similar to the existing implementation:
# Formula Functions are stored as plugins, so that new functions would not necessarily force a full rebuild of PCGen.

==Changed Characteristics==

Major characteristics of this system that are major modifications of the existing implementation:
# Formats other than numbers are supported.
# Starting value for a variable is based on the variable format (e.g. zero for numbers), so multiple defines do not "compete" [new: Define does not have a definition per DEFINE: token, but rather a DEFAULTVARIBLEVALUE once per game mode for a variable format (numbers being one possible variable format). This is consistent with and thus formalizes the data best practice of defining variables to zero].
# Terms are eliminated. They should be replcaed by functions or user variables.
# BONUS:VAR is eliminated and replaced with Modifiers (see below) [and in general ALL BONUS tokens will eventually disappear].
# The new system will not have any terms. All items will be a function or a user defined variable. This will dramatically simplify dependency analysis.
# The system does "integer aware" arithmetic, meaning 2+3=5 (not 5.0 or 4.999999999 or 5.000000001 as we might have today).

==New Capabilities==
Major characteristics of this system that are new capabilities:
# Diagnostic tools in the UI should provide the ability to expose the entire calculation, from default value through all modifications (and source of those modifications).
## It is important to recognize that the data practice of indirection (e.g. one object awards a Template called FACE_20 that sets the FACE:) is STRONGLY discouraged in the new system. That indirection makes it A LOT harder to figure out what modified a variable - something that the new system can make VERY clear, but that the data can hide and make difficult.
# Formulas are validated at data load to ensure good syntax, and that any functions that are used exist (and are valid) and variables are nominally defined somewhere in the data.
## There are a few rare excpetions to full validation, because they really occur at runtime... so just like we can't automatically detect that a divide by zero will occur, we can't detect that a Table provided to a lookup() function will actually have the necessary values and columns for the incoming variables.
# Dependencies are tracked and any circular logic will be identified as such during calculation (it is expected that this is not possible to catch at LST load due to formulas on impossible data combinations leading to false positives). This is done in the specific Solver.

==Modifiers==

The MODIFY and MODIFYOTHER tokens replace BONUS:VAR. These provide a significantly enhanced experience from using BONUS, specifically:
# Specific values calculated by the formula system
# No implicit stacking rules. All stacking is explicit by the data owner
# No implicit replacement rules. All replacement is explicit by the data owner
# Dependency will be explicit and calculation will be carefully managed. (A graph of dependencies between variables is maintained by a SolverManager)

==Single Pass Evaluation==

Unlike JEP where a pass is made to determine the variables and then load those into the object, the new formula system is capable of doing an evaluation in a single pass.

Having a single-pass resolution by a visitor to the tree, with the context passed in as a parameter, has a number of functional advantages:
* This effectively makes a formula "immutable" in the sense that the context is passed in during resolution and thus can be evaluated in multiple threads without causing issues. Note this is a necessary consideration if we want to reuse formulas that are in the data multiple times, as the UI demonstrably triggers evaluation of items from multiple threads. The visitors are also "immutable" (they are in fact, fully stateless), and thus reusable without being concerned about thread safety.
* This keeps the formula as light-weight as possible (The tree structure is still a bit expensive, but better than a PJEP).
* The context can be set based on what is passed in, meaning evaluation locality is driven by the caller (knowing the locality) rather than forcing the formula to evaluate some string to figure out where it is.
* The entire concept of a PJEP pool goes away (in exchange we effectively have a context, but those are reusable so we don't ever have the lock/free necessity of PJEP). (For specificity, those solving contexts are called "DependencyManager", "EvaluationManager", and "FormulaSemantics", and can currently be found in the PCGen-formula library.
* We actually prohibit any funky hoop-jumping. By only providing the context in the sense of a variable library, we can accurately handle things like spell-localized items (e.g. today's CASTERLEVEL term) without having a temporarily set global item.

=Further Reading=

For more detailed information on setting up the new formula system, see [[Setting up the new Formula System]]

Index to Architecture Documents

2018-03-04T21:06:47Z

Tom Parker:

{| align="right"
| __TOC__
|}

This is effectively a "shorthand" list of documents (and a tiny bit of context around the links) for "A Day in the Life of PCGen"

=Resources=

Before reading about the details of the PCGen architecture, it is likely helpful to understand our key [[Design Concepts for PCGen]] and [[Architecture Document Conventions]]

=Loading=

# The [[Startup System]] initializes PCGen and loads the initial user interface
# Files from Disk are loaded by the [[Rules Persistence System]]
# These files use [[CDOM References Concept Document|CDOM References]] to resolve order of operations conflicts during load and [[Referring to Groups in LST Data|Refer to Groups in LST Data]] to identify multiple possible objects (for say, a choice)
# The [[Load Commit Subsystem]] is used to commit information into the [[Rules Data Store]]
# Post Processing is completed as described in the [[Full Load Order Detail]]

=Using Data=

# When retrieving items from the Rules Data Store, it's important to [[Identifying Objects|Identify the right Object]]
# You will also need to know about [[Accessing Information]]
# And strategically, it's good to know how we are [[Becoming Data Driven]]

=Processing a PC=

# After [[Adding items to the PC]], we begin [[Calculating Items on the PC]]
# This calculation heavily depends on the [[Formula Systems]]
# Some of the items on a PC can be conditional, so it's valuable to understand [[Prerequisites and Requirements]]

Becoming Data Driven

2018-03-04T21:06:45Z

Tom Parker: Created page with "{| align="right" | __TOC__ |} =Input= ==Historical/Original== Originally PCGen had a series of tokens that imported information into specific locations within the objec..."

{| align="right"
| __TOC__
|}

=Input=

==Historical/Original==

Originally PCGen had a series of tokens that imported information into specific locations within the objects within PCGen. These objects were highly specialized and in somecases very long (PCTemplate was at one point well over 10,000 lines of code).

In the long term, however, we end up in a situation where the disadvantages outweigh the clarity of this architecture, so this design is deprecated and largely removed from PCGen. (The major exception is Equipment, where you can see this design in full use).

===Defaults===

The hardcoding to specific methods made defaults fairly easy to understand - they were hardcoded in the specific method, if no other value was set by the data.

===Limitations===

This hardcoding has some limitations:

# The original designs were directly aligned to how things were originally in the SRD (3.0) and RSRD (3.5). The result was that certain features could only be reached from certain objects.
# The original designs presumed certain features, meaning that it was assumed that "Hands" was a relevant characteristic. In a system like MSRD, where all creatures are human, those methods are either unused, or worse, have to be turned off so they aren't processing and checking for data that may not be present.
## Alignment is a specific example where there is specific code that has to turn OFF certain features if there is not an Alignment in a specific game mode
## This turn on/turn off is inflexible and really not scalable.
# In general every new characteristic not only drives a new token but it drives a new set of methods on the underlying objects.
# From an output perspective, specific output tokens must call the exact methods, so significant code is requried in the output system as well.

===Advantages===

This hardcoding is nice for certain things:

# It makes it clear to a reader what is being processed, since there is a direct method on an object of, for example, getHands()

==Indirect Storage==

As we began a transition in reducing duplicate code and making PCGen more flexible, we got to where many of the items are now resolved indirectly. Instead of getHands() for example, we now do: getInteger(IntegerKey.HANDS). These generic storage methods are on CDOMObject.

This has a number of benefits, specifically that adding additional characteristics to an object does not require new methods (there is an ability to store integers, Objects, Lists, etc.). However, a new token is still requried, since there is not a generic method of getting at those put* and get* methods from the data tokens.

The Keys for these indirect storage methods are generally in pcgen.cdom.enumeration

===Defaults===

The lack of specific methods for any given behavior then leave the question of where defaults should be stored... To provide that behavior, we allow each Key to hold a default value. Then a resolution can use the default value if no other value exists.

You can see the Defaults in places like IntegerKey, where the constructor takes 2 arguments (The name and the default value).

===Design Choice===

Note that there is a specific set of methods that handle Lists and more complex data structures on objects like Race, et al. This is conscious, in order to fully protect the contents of the list held internally to the CDOMObject from being modified outside that CDOMObject. As mentioned elsewhere, this is a defensive coding syle based on past problems in PCGen and hedging against similar issues.

==Indirect with a Generic Token==

The next progression in improving how PCGen can be data driven was enabling a token that allowed the Data team to not only define their own content, but define their own "Key" as well.

To do this, there are two new tokens, FACT and FACTSET. Much of the rest of this section refers to "FACT", but is using that as a generic term for both FACT and FACTSET. Few features only support FACT, and in those situations, it will be clearly stated.

===Design choice: Predefinition===

Note that it would be possible to simply have a generic method that would automatically generate a *Key and inject objects using the same methods as in "Indirect Storage" above. However, this could be highly error-prone. A small typo of "Hadns" would result in data never actually controlling the right value. We therefore made a conscious decision to force the pre-definition of the key values. This is done through FACTDEF and FACTSETDEF tokens in the Data Control file.

===Design Choice: Static===

An almost immediate response upon the introduction of FACT and FACTSET was: "Why can't I change them". Answer: Facts shouldn't change, so semantically the behavior matches the name.

Well, on a more-useful answer (of why FACT in the first place), there are practical reasons to have them not be modifiable. The first is that we encounter many items that are not-PC specific that can therefore be cached. To guarantee we can cache that information, we need to know we will not be PC dependent. The design of this makes that guarantee.

Second, this was design was done in context to the new variable system being in the design pipeline, so we clearly didn't want to duplicate efforts of a variable system with local variables. If the data team needs to modify something, then clearly it's a local variable, not a FACT.

===Data Driven Grouping===

When a FACT is provided, if it is something that is known to be universal (or nearly so) it is possible to use that to group objects. Groups are usable in places like CHOOSE:. To enable the use of a FACT in a Grouping situation, it needs to be enabled, by using the GROUPABLE: token on the FACTDEF.

===Data Driven Output===

A FACT can also be enabled so the information is visible to output. This is also controlled on the FACTDEF line.

=Output=

Traditional output was done using a custom PCGen output system. Some time ago, we adopted Freemarker, and many of our output sheets now use Freemarker to do output

==Traditional==

Traditional output uses the Tokens generally in plugin.export. A number of them are in the core, in pcgen.io.exporttoken. These items process information read from the output sheet and process it into the appropriate String.

===Challenges===

# Many of these tokens take a significant amount of processing and keep it within PCGen. The weapon and ability output tokens both are extremely large.
## The ability tokens require a cache to have reasonable performance. Because of how the traditional output system works, the tokens become re-entrant, and have to continually build the same ordered list. Since that is not in the core, some of the tokens cache the list and it can be confusing about why the cache exists to an unaware reader of the code.
## The weapon tokens have a ton of processing, much of it very specific to d20 systems.
# The tokens themselves are not at all modular - they are all generally independent and are a huge source of duplicated code. Others have to be held in the core so that derivative tokens can have them as a parent class.
# The traditional output tokens can be used in the traditional variable system as a value.
## You probably can't imagine how difficult this makes following what is possible in the old variable system in terms of how much code will be called when a formula is processed

==Freemarker Compatible==

As we transitioned to using Freemarker, we were able to stop using a number of the control tokens we used to have (IF, etc). Others we continued to use in a fashion that was very similar to their existing usage. This still carries with it many of the existing issues articulated above.

==Freemarker Native==

Going forward, we are moving toward a Freemarker-native form of output. You can find this in pcgen.output. Instead of doing custom String processing relative to PCGen, these classes provide TemplateModel objects that are fully compatible with Freemarker's internal knowledge. This will provide a number of advantages:
# Since the values are effectively native to Freemarker, they will require no caching, et al. (Freemarker will capture and reuse the list while it is in scope)
# There is a huge ability to be modular in terms of reuse when certain things show up in different places.
## For example, the PC's Alignment may desire certain features (showing the abbreviation, etc.). This may also be true of the Deity's alignment. Under the old system, this would have required duplicate code or some very clever hoop jumping in the old tokens. In the Freemarker native system, we well Freemarker how to wrap an Alignment, get back a TemplateModel, and then regardless of the method used to get to an Alignment, we can provide all of the features of an Alignemnt all of the time. IT naturally reuses code.

===Challenges===

Right now a limited set of items are available in Freemarker-native output. Some of this is conscious, since the desire to bring across a lot of the custom code in things like weapon output tokens is low... we want to drive those into the data and into the new formula system.

=Design=

The various methods of input and output hav esome consequences going forward. One of those in understanding what choices to make in being data driven is in using Variables vs Objects.

The example used here relates to Movement.

It is definitely possible to have new variables like:
<pre>
Movement_Normal
Movement_Fly
</pre>

We could perhaps start these at -1 to indicate no movement is available, or even have boolean values like:
<pre>
Has_Movement_Fly
</pre>

However, this quickly breaks down for a number of reasons:
# It gets very difficult to increment across all Movement.
## The output sheets will need to do that, and if it was strictly variables in the new system, the output sheet would need to check all the variables.
## This potentially traps all games modes into using the same variable names (and potentially other dynamics), which is a bit rediculous. If data driven means there are handcuffs on the data, it may be worse than having magical code.
# It hardcodes values like "Fly" into variable names, and those are things that should eventually be localized (meaning translated)
## This would thus demand unique infrastructure for translating variable names
# An item that attempted to modify all movement of any type would have to know all the variables and would thus be fragile (meaning you'd have to constantly be modifying that object).
## This indicates an underlying architecture failure of the design if a simple change results in breakage elsewhere.

To get around these items, it is recommended that items of this nature (Movement and Vision being the most obvious) use a separate object type:
<pre>
DYNAMICSCOPE:MOVEMENT
</pre>

This then gives an object type of "Movement", and new items can be defined:
<pre>
MOVEMENT:Normal
MOVEMENT:Fly
</pre>

Now we end up with a few neat characteristics:
# Since these are objects, we can still focus on translating info about objects and we will thus capture and be able to define how to translate these
# We can have a local variable on each item, and thus address that from various locations
## It's no longer a series of global variables, so we have reduced the number of global variables
## If we have something that ever modifies all movement of a given type, it is as simple as a MODIFYOTHER:MOVEMENT|ALL|... so it's not fragile to introduction of a new type of movement.
# We can easily increment across all objects of that type. It can easily be exposed on "pc" as something like "movement", and thus flows directly into the same process as other items that are Freemarker-native output.

SELECTION - A CHOOSE Replacement

2018-03-04T19:13:05Z

Tom Parker:

{| align="right"
| __TOC__
|}

=Introduction=

First draft proposal for SELECTION (replaces CHOOSE/supports multiple CHOOSE).

Please note that the token names here are for example purposes. Some discussion has taken place but they are not
finalized. They are here to show the concepts and how the structures of this new system would work in LST files.

=Code Concepts=

==Current Issues==

===Multiple Methods===

There are currently 4 different methods for how CHOOSE operates in the code
* Abilities
* Race and Template
* Language
* Temporary Bonuses

The desire is to reduce this to one methodology (or two at worst if temporary bonuses need to be separate)

===Single Selection===

CHOOSE currently supports only one selection. In many cases this produces a challenging format that is forced into the
code. If multiple selections were allowed, this would not be necessary, as one could be CLASS and the other LEVEL for
example.

===Automatic Expansion===

CHOOSE currently automatically expands items. If you CHOOSE:FEAT, you get things like Weapon Proficiency(x) and Weapon
Proficiency (y). This also then creates an interesting visual challenge if there are multiple tiers. If a Feat allows
a selection from another MULT:YES feat, then you get Something(a(b)) and Something(a(c)). This can rapidly proliferate
into a huge list to scroll through. It would be visually cleaner to select "a" and then select "b or c" within the
context of "a" being selected under "something". This requires a new UI to handle this hierarchical behavior.

(This item is not currently addressed further here, as I would like to talk to some of our UI folks before I propose
too much here)

===CHOOSE:NOCHOICE===

Simply shouldn't have to exist.

===Stacking Ambiguities===

This is true more for MULT:YES than CHOOSE in particular, but imagine something like this:

ABILITY:FEAT|VIRTUAL|Dodge

Now put that on a MULT:YES Feat and select it twice.

Do you get 2 copies of Dodge or 1 copy?

This has already been encountered in the past and discussed on the -dev list. The BONUSes also suffer from the same
issue and there have been a few debates over proper behavior.

The solution to this problem is eliminate all ambiguity. We should to distinguish between the ability itself (applied
only once ever) and what I will call the "instances" of the ability (applied once per time taken by the user). Note
that I do NOT say "per selection" or "per choice" because if an item has 3 choices, we still have struggles. There are
ways to apply once per choice as well, but we need ALL of the capabilities, so we have to have a method to do once per
time taken by the user).

From this point onward, I will have an INSTANCE: token that then refers to instances rather than the parent object.
This is much like the current "PART:x|..." token that can appear on equipment to handle equipment heads. Just like
PART, it will have an argument (as to where it is applied) followed by another full token.

==Current Quirks==

===Stacking===

Currently the system defaults to STACK:NO. Should this be the default behavior when a lot more control exists for the
data to define what is selectable?

===Race Conditions===

Part of the reason we have multiple different selection methods is that the code has certain race conditions (two
actions that need to happen in a certain order).

For Race and Template, they get added to the PC first, THEN the selection is made. This means they have to be
defending against undefined choices.

For Abilities, they only get added to the PC when an AbilitySelection object (an ability with its selection) is added
to the PlayerCharacter. This means the selection occurs BEFORE the object is added.

==Other Considerations==

===Conversion===

No conversion will be automated, as the semantics here are completely different than the old system.

Also, since backwards compatibility will not be supported (CHOOSE will not be available in the new formula system at
least under the current plan), we will need a new token name. From this point onward, it will be called SELECTION.

==Design==

===Instances===

As mentioned above, the Selections will be placed into instances. These instances are separate objects with a separate
SCOPE. For Skills (For example), the scope will be SKILL.INSTANCE

===Aligning Concepts to the new Formula System===

In general, the new formula system uses the concept of a Scope to determine where a variable is processed.

A naive design would be to keep two lists on the main object. That way, the first set of choices are [0] in the lists,
the next set [1]. While I don't have a specific rule associated to this, I can quickly come up with some plausible use
cases where that would break. For example: Imagine a feat that allows you to select a class and make 2 skills from the
classskilllist of that class get +1 bonus rank.

Therefore, we delegate containing the result of the selection down into an instance object. We then make that a
subscope of the main scope for that object type. Consider the CHOOSE on skills for Speak Language. We then get
something like:
INSTANCE:ALL|SELECTION:SpokenLanguage|LANGUAGE|getAll("LANGUAGE")

Now, every time a new instance of that Skill is selected (which happens to be from getting a new rank of that skill), a
new Instance will be added to the PC and a new selection will need to be made.

It could then be added to a list of languages via:
INSTANCE:ALL|MODIFY:LanguageList|ADD|getSelection("SpokenLanguage")

(This assumes LanguageList is a global variable of Format "ARRAY[LANGUAGE]")

To be VERY CLEAR: getSelection("SpokenLanguage") MUST BE USED IN AN INSTANCE. If you attempt to use it up at the
parent ability/skill/whatever, it will either (a) fail or (b) be empty. (It should fail, and we will endeavor to do
that, but there will need to be enhancements to have functions available only in a limited scope... that's new effort)

===Resolving the Race Condition===

The race condition is particularly problematic for the new formula system. Consider a naive design like this:

MODIFY:MyList|ADD|Foo
SELECTION:MySelection|STUFF|myList

This immediately sets up a problem based on how the formula system works. Because it only processes items that are
actually "granted" to the PC, the MODIFY of MyList would not happen unless the object itself was already granted. This
traps us (a bit) in making sure the object is added to the PC BEFORE a selection is made, so that if it chooses to do
any list modifications, they can be done BEFORE the selection.

To resolve this race condition, we put the MODIFY on the parent object, which is granted BEFORE any selection. We then
put the SELECTION in the INSTANCE, which is granted AFTER any selection. Note this also means that if (for some
reason) the data team to have the MODIFY occur AFTER the selection, they have that power. This gives the data team
full feature, supporting add both BEFORE and AFTER selection:

MODIFY:MyList|ADD|AddBeforeSelection
INSTANCE:1|MODIFY:MyList|Add|AddAfterSelection

As a nice side effect, this actually gives us full power to avoid ANY popup windows in a new UI should that be a
feature we want to implement. Because all instance applications can be deferred, they could be "TODOs" on the PC
rather than stopping for user input.

===Replicating NUMCHOICES===

To replicate the behavior of NUMCHOICES, we then need to control the number of instances that we are allowed to apply.

This is a new token, MAXINSTANCES.

===Replicating SELECT===

To replace the behavior of SELECT, we need to control the number of actual selections that exist in an instance for
that SELECTION. Since this is associated to the SELECTION, we need to keep it local to that token, so it is appended
as an optional control on the SELECTION:
SELECTION:MySelection|LANGUAGE|getAll("LANGUAGE")|SELECT=2

===Happy Side Effects of INSTANCE===

Please note: ALL objects that are instance-able ALWAYS will produce an instance. This is true REGARDLESS of whether
there is a SELECTION on the object. Many of these Instances will be empty shell objects, but they can also have some
interesting uses.

"At Rank 5 this skill adds 2 legs to the PC"

INSTANCE:5|MODIFY:Legs|ADD|2

Note that this means there is no equivalent PRERANK: needed on a MODIFY. It is directly applied to the "5" instance as
a MODIFY, and when the Rank hits 5, that INSTANCE will be granted and the MODIFY will be applied. (If you are thinking
this could be useful for how pathfinder handles Skill Focus... yes, possibly... stay tuned :) )

===Using a Selection as a Target===

In some cases, we end up in situations like "This will increase the range of one type of weapon by 5'". Today there is

BONUS:WEAPONPROF=%LIST|RANGE|5

The %LIST here is actually in the target, which is a bit of a problem! The second argument to MODIFYOTHER is
definitely NOT a formula, so we need to address that somehow. In the future, there may be elegant ways of doing this. For right now, we end up with a not-so-great method, which looks like:

MODIFYOTHER:EQUIPMENT.PART|ALL|Range|ADD|if(getFact(parent(),"WEAPONPROF")==getSelection("MySelection"),5,0)

=Data Information=

==New Tokens==

===INSTANCE===

INSTANCE is only supported on object types that are INSTANCE-able. This will require code support for all existing
Formats.

Note: It is the intent to allow DYNAMIC to be instance-able if the data team so specifies. More on this later.

Format:
INSTANCE:x|y

* x is the instances to which the token given in y should be applied. Currently this supports either an integer value or "ALL"
* y is a full token to be applied to that instance.

Please note INSTANCE objects are NOT full CDOMObjects. They cannot support arbitrary tokens. Only a limited set
(defined here) will be supported.

===SELECTION===

SELECTION will ONLY be available on INSTANCE objects. This means it needs to appear as a subtoken to INSTANCE (which
itself must only appear on instance-able objects).

Format:
SELECTION:w|x|y|z|z

* w is the name of the selection. This is relevant when the data is using the value of the selection in the data
* x is the FORMAT of the selection. See the [[Setting up the new formula system]] for more information on formats.
* y is the array of available objects for the selection.
* z is an optional set of controls for the selection.

====Selection Control: SELECT====

This replaces the SELECT: token in the current CHOOSE System. It is a z value for SELECTION:
SELECT=x

* x is a formula for the number of selections to be made.

====Selection Control: ORDER====

In the scenario where two SELECTION items would interact (such as choosing a level of spell for a given class, where
the class would need to be selected first), there is an ORDER control:
ORDER=x

* x is an integer constant. The SELECTION with the lowest ORDER will be processed first (just like PRIORITY on
MODIFY). If two SELECTION items have the same ORDER, the system will not guarantee which is processed first. If it
matters, specify it. The default value is zero.

===getAll(x) Function===

getAll will be a function in the new formula system.

Format of this function:
getAll("x")

* x is the name of a FORMAT from the new formula system.
** Note: This format MUST be an limited format, in that you can't do getAll("NUMBER") as that is an unbounded set.

Note also this function will take some cleanup - we eventually need to figure out how to do the things that the
existing items in the CHOOSE system do... this is a patch for now to get the discussion started

===getSelection(x) Function===

getSelection will be a function in the new formula system.

Format of this function:
getSelection("x")

* x is the name of the selection from the SELECTION token.

NOTE: The FORMAT that getSelection will return will depend on the SELECTION token. It also depends on whether there is
a SELECT=x control on the SELECTION

===MAXINSTANCES LST Token===

This token will be usable on any format which is instance-able.

Format:
MAXINSTANCES:x

* x is a formula in the new formula system to identify the maximum number of instances allowed for that item.

=Issues=

# The current design of tokens shown for instances assumes one instance type per object type, which is actually not true
# The current design does not consider how to process ALLOWED/ENABLED for selections - is that a filter, or not - it should be clear and under control of the data team

Index to Architecture Documents

2018-03-04T18:23:04Z

Tom Parker:

{| align="right"
| __TOC__
|}

This is effectively a "shorthand" list of documents (and a tiny bit of context around the links) for "A Day in the Life of PCGen"

=Resources=

Before reading about the details of the PCGen architecture, it is likely helpful to understand our key [[Design Concepts for PCGen]] and [[Architecture Document Conventions]]

=Loading=

# The [[Startup System]] initializes PCGen and loads the initial user interface
# Files from Disk are loaded by the [[Rules Persistence System]]
# These files use [[CDOM References Concept Document|CDOM References]] to resolve order of operations conflicts during load and [[Referring to Groups in LST Data|Refer to Groups in LST Data]] to identify multiple possible objects (for say, a choice)
# The [[Load Commit Subsystem]] is used to commit information into the [[Rules Data Store]]
# Post Processing is completed as described in the [[Full Load Order Detail]]

=Using Data=

# When retrieving items from the Rules Data Store, it's important to [[Identifying Objects|Identify the right Object]]
# You will also need to know about [[Accessing Information]]

=Processing a PC=

# After [[Adding items to the PC]], we begin [[Calculating Items on the PC]]
# This calculation heavily depends on the [[Formula Systems]]
# Some of the items on a PC can be conditional, so it's valuable to understand [[Prerequisites and Requirements]]

Accessing Information

2018-03-04T18:23:01Z

Tom Parker: Created page with "{| align="right" | __TOC__ |} =Levels of Information= As we have separated information inherent to data from data related to a specific PC, we have had more ability to s..."

{| align="right"
| __TOC__
|}

=Levels of Information=

As we have separated information inherent to data from data related to a specific PC, we have had more ability to separate out information and keep it at the level where it is defined. To do that, we now end up accessing information within different scopes.

==Game Mode==

Some information is set in the Game Mode, and we store/access that information by Game Mode. Game Modes are loaded once when PCGen is booted, and not reloaded. Therefore, this information is generally static/persistent information.

==Data Set==

When a set of campaigns is loaded by a user (or implied by opening a specific PC), we call that set of Campaigns a "Data Set". Many items (list of available Races, Languages, etc.) are inherent to the Data Set. Much of this information will need to make its way out to the UI to be displayed to the user. For the most part, this information is stored today in the LoadContext and its children.

==Player Character==

When a PC is created, there is a set of information related to the PC. This information is stored in various places, but in general it is stored within the Facets and new formula system. More detail is available in [[Adding items to the PC]].

=Architectural Musings=

This is one of the areas where the default design methodology of Dependency Injection (DI) doesn't work well if you consider the base design of a DI system (pre-wiring objects). While the Facets themselves today are static, MANY of them are dependent upon assumptions about the data we load, and the fact that many of our game systems are d20 systems. Some of them are idle Facets (and thus shouldn't have been loaded under ideal circumstances) for non-d20 systems. This area needs some help.

There are also aspects of DI systems that are highly interesting given the different scopes of information shown above. The ability to define an information "scope" (in the case of Spring an org.springframework.beans.factory.config.Scope), is definitely along the lines of what would help keep information loadable one time while "guaranteeing" that the scopes of information cannot leak. This would be of significant benefit to a desire to load multiple systems or multiple PCs at the same time without the significant complexity we currently bear from that situation.

After having read through how the HTTP (request/session) scopes work in Spring, I find it highly intriguing to implement a similar design. My primary concern is that I'm not sure anyone currently active really understands the multiple threads generated by the UI, which could cause a significant amount of damage if not handled correctly.

More on this later...

Adding items to the PC

2018-03-04T17:42:06Z

Tom Parker:

{| align="right"
| __TOC__
|}

In the process of adding items to a Player Character, an Object (like a Language) can be in a number of possible states.

First, it can be "Available". This is possible with things like the list of Bonus Languages, Class Skill Lists, and Class Spell Lists. Things can also be prohibited with items like [[Prerequisites and Requirements]].

=Granting=

When an object is attached enough to the Player Character in order to change that PC, we refer to that as "granting" the object to the PC. Items like stats (PCStat objects) are universally granted; Skills are granted if they have at least one rank; other objects like Race are granted directly.

=Spell Life Cycle=

Spells are not granted, they have a different life cycle, which involves known and memorized.

=Selections=

Certain types of objects may need a choice to be made as they are added to a PC. Abilities, for example, use a CHOOSE: token and MULT:YES to indicate this situation. Templates and Races use CHOOSE: (no MULT: allowed). Equipment Modifiers also can use a different set of CHOOSE: tokens.

==Template/Race==

When a Template or Race is added to the PC, it is checked to determine if it has a CHOOSE: token. If it does, a chooser is fired, and the user is forced to make a selection.

It is currently not possible to assign the result of a CHOOSE: to a Race or Template when applying the Race or Template in a non-interactive fashion. (Meaning you can't do TEMPLATE:MyTemplate(ChooseResult) and expect it to make the ChooseResult selection from the CHOOSE: token... Templates are NOT Abilities)

===Warnings around use===

Note that there is some inherent danger in this design: If the system is trying to add a Race or a Template to a PC in a moment when it is not interactive with the user, then the result is not defined, and may result in a crash.

Note also with CHOOSE: on Race that certain information may not yet exist in the PC in order to make certain choices available. This is a race condition we do not intend to fix, as in many cases, the question tryign to be answered cannot be rationally answered at that point in the code.

===Storage===

When a selection is made, the result is stored as an association to the Race or Template. From a code perspective, it is effectively a Map<Race, Object> or Map<PCTemplate, Object> (although the actual location of the association is in a Facet, e.g. RaceSelectionFacet).

==EquipmentModifiers==

When an EquipmentModifier is interactively added to a piece of Equipment (in the Equipment Builder), the user is forced to make a choice, if necessary.

When an EquipmentModifier is added through the EQMOD tokens on Equipment, then the result of the selection is provided in brackets in the token, e.g. "MyEqMod[MySelection]".

===Storage===

Since Equipment is still cloned between instances, EquipmentModifiers are stored on the Equipment. Technically, they are actually stored on the EquipmentHead objects, since the modifiers like +1 (or whatever) are technically attached to the Head, and not to the entire piece of Equipment.

==Ability Selections and Ability Life Cycle==

We start with an "Ability". An "ability" is a named entry in an Ability file.

If the Ability has a chooser within it, the product of executing the chooser on the Ability is an "AbilitySelection", which contains the result of the choice. If it doesn't, there is still one default "AbilitySelection", with a null choice. There are both UI and LST data file methods of getting to an AbilitySelection - these are described below.

Once a Category and Nature are added, we successfully have a CategorizedNaturedAbilitySelection or CNAS. That can be applied to a PC.

===An Example on Terminology===

Let's assume there are 2 TYPE=DragonForm abilities:
<pre>
BasicDragonForm <> TYPE:DragonForm <> MULT:NO
AdvancedDragonForm <> TYPE:DragonForm <> MULT:YES <> CHOOSE:PCSTAT|ALL
</pre>

There are 2 abilities here:
<pre>
BasicDragonForm
AdvancedDragonForm
</pre>

There are 7 AbilitySelections here:
<pre>
BasicDragonForm()
AdvancedDragonForm(STR)
AdvancedDragonForm(INT)
AdvancedDragonForm(WIS)
AdvancedDragonForm(CON)
AdvancedDragonForm(DEX)
AdvancedDragonForm(CHA)
</pre>

Note I'm writing even MULT:NO stuff with a trailing set of () to try to distinguish the AbilitySelection from the Ability. The user never sees that thought process (it's internal to the code)

There are many combinations of CNAS that are possible; they are not all articulated here. (There are, at a minimum, 21 possible combinations. 3 natures for each of the 7 AbilitySelection items listed above assuming there are no child Category objects for the Ability... 7 more for each child category where these items could be selected [child categories only use NORMAL nature].)

===Granting an Ability to a PC===

Abilities are not directly granted due to their structure. In order for an Ability (e.g. Feat) to be applied to a PC, it requires:
# A Category (either the parent Category or a child Category - in this case to indicate which "pool" is charged for taking the Ability)
# A Nature (Virtual, Automatic, Normal)
# The Ability itself
# Any Selection on the Ability. The Selection is the result of a CHOOSE token being present on the Ability. If no CHOOSE is present, the the selection is null, but still required to be present in order for an Ability to be applied to the PC.

As much as this might make the data team's head hurt, at the end of the day a character can never have an Ability applied to the character. It's always a CNAS. It's just a matter of how you get the full CNAS.

Usually it starts with an AbilitySelection. When combined with a specific Category and Nature, the AbilitySelection can be granted to a PC. (a CNAS can be granted)

===From the UI===

Even when you select BasicDragonForm from an Ability Pool or the Feat screen or whatever, the system internally evaluates it's MULT:NO and expands it to BasicDragonForm() to get the AbilitySelection. When you select AdvancedDragonForm from a pool, the system evaluates that it's MULT:YES, determines there are multiple possible answers, and makes you pick before it has the AdvancedDragonForm(target stat).

Either way, you have generated an AbilitySelection. The screen/pool you are in defines the Category, and the nature is inherently NORMAL for items selected in the UI - so the system can generate the full CNAS to be applied to the PC. It's just that most of the work is transparent to the data team.

===In LST Data===

CHOOSE:ABILITY returns an Ability:
<pre>
BasicDragonForm
AdvancedDragonForm
</pre>

So at any point, I can select an Ability with that... but the only thing I can GRANT to a character (what ABILITY: does) is an AbilitySelection.

Form of the Dragon II
<pre>
CHOOSE:ABILITYSELECTION|Special Ability|TYPE=DragonForm
ABILITY:Special Ability|AUTOMATIC|%CHOICE
</pre>

This needs to be CHOOSE:ABILITYSELECTION and not CHOOSE:ABILITY because CHOOSE:ABILITY selects an ABILITY and ABILITY: applies a CNAS (so the %LIST must provide an ABILITYSELECTION).

====An Example showing the subtlety====

To understand the distinction, consider a Feat like Martial Weapon Proficiency.

The Ability is Martial Weapon Proficiency. This appeared in the LST file as an Ability.
Applying the first to a character is not meaningful (and reintroduces bugs we had with Martial Weapon Proficiency in the past that we fixed by introducing CHOOSE:FEATSELECTION).

An AbilitySelection is Martial Weapon Proficiency (Longsword). This was the result of the CHOOSE:ABILITYSELECTION and is the piece of information transferred from the CHOOSE to the ABILITY token
Applying the second might be possible, but would not know what pool to deduct the application from or where it should be displayed in the UI (or in what color)

A CNAS is FEAT::NORMAL::Martial Weapon Proficiency (Longsword). This was the result of combining the AbilitySelection with the other information provided in the ABILITY: token.
The last one (which is effectively generated from all the information provided to the ABILITY token) is grantable (because that's what the ABILITY token does).

===A Countercase to using CHOOSE:ABILITY===

From an older conversation on pcgen_experimental:

The concern: If none of the abilities-of-interest have choosers within them (so they are like your BasicDragonForm example - in fact, they are BlackDragonForm, BlueDragonForm etc.), it's counterintuitive that I can't grant the ability from the CHOOSE:ABILITY result, as it seems a small leap to the non-coder from the ability BlackDragonForm to its abilityselection BlackDragonForm()!

Let's walk through the scenarios:
<pre>
CHOOSE:ABILITY|Special Ability|TYPE=DragonForm
</pre>

Let's assume you start with all of the TYPE=DragonForm items being MULT:NO, so you can infer the AbilitySelection from the Ability. One could assert:
<pre>
ABILITY|Special Ability|AUTOMATIC|%LIST should work.
</pre>

Now someone comes along and adds a MULT:YES item. What do we do?
# Error on data load
# Don't let the CHOOSE:ABILITY present it
# Let the user select it, but it never gets applied to the PC.

In case #1, the code needs to recognize that a catcher (%LIST) is asking for an AbilitySelection, and a thrower (CHOOSE) is producing an Ability. Then it needs to check for auto conversion. At each load, all items of that TYPE (or listed individually) need to be checked to ensure they are MULT:NO. So we've spent a bunch of time iterating across the data.

Also, we can produce some nasty cases here. Imagine that a data monkey puts in "hollow" Abilities with only a DESC (which will happily take either Ability or AbilitySelection) to produce output in order to test their data. It has MULT:YES. The system will work with CHOOSE:ABILITY. They then go back and put in an ABILITY token to "do it for real" and the data breaks. Well, "obviously" that's a code problem (when in fact, they have changed what they are targeting).

So this scenario is (a) costly to enforce (b) fragile [meaning it produces non-intuitive errors and small extensions (creating a new Ability) will cause errors to be reported on unrelated objects]

Case #2 suffers from similar issues except that the checks are performed at runtime. If I ask for TYPE=DragonForm in CHOOSE:ABILITY, I expect it to be TYPE=DragonForm not "TYPE=DragonForm if and only if it's MULT:NO". The latter to me is what I would refer to as "magic" and something I generally try to avoid.

Case #3 also seems to suffer some nasty side effects. "I selected BasicDragonForm and it applies, but AdvancedDragonForm won't apply (and by the way, as part of your "code bug" it didn't provide me the ability to select my stat). I really don't want these "support calls" to the code team to fix things that aren't broken.

So we are strict. When a CHOOSE produces an Ability, you can do something with an Ability, when a CHOOSE produces an AbilitySelection, you can do something with the AbilitySelection. We are getting to the point where we actually produce errors about that today if you try to target the wrong type. (Go try CHOOSE:STAT|ALL and FAVOREDCLASS:%LIST in a Template and see what happens... 6.0.0 will silently consume that data. 6.1.x-dev, not so much :) )

===Another example using CHOOSE===

Reviewing:
<pre>
ABILITY:FEAT|AUTOMATIC|%LIST wants to "catch" an AbilitySelection.
</pre>

Using the model where parens indiate an AbilitySelection, this is attempting to catch something of the form:
<pre>
???(?!?)
</pre>

Note the ?!? may be null in the case of MULT:NO, but there may be 2 unknowns.

This case:
<pre>
ABILITY:FEAT|AUTOMATIC|Weapon Proficiency (%LIST)
</pre>

This wants to "catch"... well, we don't necessarily know without looking.
What we do know know is the Ability, it's called Weapon Proficiency.
So our AbilitySelection is effectively "Weapon Proficiency (?!?)"

What we need to fill in is a single unknown, the Selection.

CHOOSE:WEAPONPROFICIENCY chooses a Weapon Proficiency. So we are "throwing" a Weapon Proficiency... but are we catching the right thing?

What the code has to do is go peel back at the Feat Weapon Proficiency.
<pre>
Weapon Proficiency <> MULT:YES <> CHOOSE:???|...
</pre>

??? had *better* be WEAPONPROFICIENCY, so that the "catcher" is catching the appropriate type. If not, an error should be thrown at data load.

If the CHOOSE *does* match up, then effectively the ABILITY token in this case catches the Selection, plugs that into the Weapon Proficiency to produce Weapon Proficiency(known selection) [aka an AbilitySelection] and can grant that to the PC.

So another way of looking at CHOOSE:ABILITY is that is answers one unknown: The Ability.
The CHOOSE:ABILITYSELECTION is actually a hybrid CHOOSE that (somewhere under the covers) answers two questions at once, effectively giving you the Ability and the Selection in one choice.

=Pools=

In some cases, numerical items are placed into pools. Skills, and a few other items work this way. This can be rather challenging when items are subsequently removed from a PC, since we don't necessarily associate which point added to a pool resulted in which item selected by the user. We want to improve this awareness (and need to do so in order to resolve several bugs).

Longer term, this is going to be a problem, as we will want to store those details, and the result is that the contents of the current PCG file format will be insufficient to rebuild a PC to the full internal state. This transition will need to be managed carefully.

=Old Storage=

Traditionally, the metadata and implemenation of objects was mixed. (Meaning it was very difficult to distinguish what a Race meant in the rules and what characteristics of that Race pertained to the actual active PlayerCharacter. This behavior was facilitated by cloning of all of the objects before they were added to the PC.

This resulted in a number of issues, including that the clone was not always complete, the clone was not always sufficiently deep to separate PCs from each other (so they didn't collide in updating internal objects), extra memory use and understanding of what was going on for new developers.

A significant portion of the late 5.x series of releases separated the two sets of behaviors within PCGen. However, Equipment still retains the full clone design due to the complexity of Equipment. This means the Equipment object still has a significant number of internal methods and fields that contain the information about the equipment (both from the rules and reslated to the current PC).

==Equipment and Heads==

It is noted that one of the items that makes Equipment particularly difficult to deal with is the concept of "Primary" and "Secondary" Heads on Equipment. We have instantiated these as separate objects (EquipmentHead) so that the items that are duplicated can be more easily managed, and to prepare for future expansion.

This internal hierarchy has multiple sets of issues associated with it, including:
# MSRD and some other sets have a desire to have more than one additional component (so an arbitrary number of heads)
# This is a more difficult design to avoid cloning, since the depth of association required is much higher

=Current Storage=

For most other objects, the rules information has been separated from information related to any given PlayerCharacter.

The dynamic information related to a specific PlayerCharacter is now stored in a series of Facet objects (pcgen.cdom.facet).

==Facets==

Simply stated, we called these "facets" because we started to run out of other words in the English language for "part" or "component" or "aspect" (since those words are all "loaded terms" when thinking about the rules we handle in PCGen).

Facets are constructed by Spring, so that they are properly initialized. Currently we do that through XML. This is currently the only real "static" part of PCGen, thus the only part where we are using Spring to load classes.

Each Facet is designed to be a rather simple service for a PlayerCharacter or for a loaded DataSet.

===IDs===

If storage is provided as part of that services, we store things based on a CharID (character ID), so that we are not using the very heavy-weight PlayerCharacter object as the key. (The goal was to transition to a very small primary key, while slowly chipping away at the amount of "stuff" in "PlayerCharacter"). Each Facet should provide one service (storing the Race, storing Skill Ranks, etc.)

Each CharID is aware of it's "parent" DataSetID, so that if Data Set/Game Mode information is required from such a Facet, it can be easily accessed if the CharID is known.

===DataSet Facets===

A number of facets store information related to a particular Data Set - more like storing information about the Game Mode than about an individual PC. When we encounter these types of information, we don't store it by the PC, we store it by the DataSet. These all have a registration occur in their init() method that registers them with the DataSetInitializationFacet for a later callback when a Data Set is fully loaded. This callback is one of the last steps in the [[Full Load Order Detail|Load Process]].

===Copying===

In addition to storage, each Facet is responsible for defining how it needs to copy information from one PlayerCharacter to another. If, for example, it uses an internal structure for storage of complex sets of information, it will need to have a method that performs a proper deep-copy if a PC is cloned. (Unfortunately, there is still something in the old-style output that requires a clone, so unfortunately, we have to be prepared to clone PCs).

===Filters===

Some facets only provide a filter service, meaning they are reducing items passed to a later Facet.

===Facet Chain===

In many cases, the Facets provide a chain of services, each providing one simple service with others providing later processing, until we settle upon the information that is actually attached to the PC. For the primary object types (Race, Templates, etc.), the items attached the PC settle into Facets within the "pcgen.cdom.facet.model" package.

The reason there is a neeed Facet Chain is because of a number of characteristics of our current data. These include (but are certainly not limited to):
# The possibility for Languages to be granted MANY different ways and wanting to ensure that they can only be removed from the PC the same way they were granted to the PC
# Need to process MULT:NO Abilities to avoid granting them from multiple locations
# Need to process STACK:NO Abilities to avoid granting the same choice from multiple locations
# Need to Handle multiple methods of spells being made avaialble to a PC.

In general, this Facet chain is set up either in dependencies in the XML file defining the Spring configuration or in FacetInitialization.

===Bridge Facets===

Note that in some cases there are facets that perform certain services (and that link other facets in their init(), which are not otherwise called (meaning they would never activate if no one ever constructs them). We call these "Bridge Facets" and explicitly contruct them once in the doBridges() method of FacetInitialization.

===Priority===

Note that there are a few situations where Facets need to learn of changes in a certain order, so a number of the Facets support a priority associated with a listener.

===Equality===

Note that many of the objects in PCGen have strange equality defined, and unfortunately, removing that equality causes bugs (so *something* is still depending on some best-practice violating equality methods). To avoid false positives, many of the Facets use instance equality for comparison. Some MUST do so in order to be able to properly process Equipment, MULT:YES items, etc.

===Trailing PRExxx===

Current/older granting tokens allow a trailing PRExxx token to appear on the grant. This produces a number of challenges, including that any change to the PC results in a need to re-evaluate every trailing PRExxx to determine if it changed. These items typically have an update(CharID) method, and you can see them called in the setDirty method of PlayerCharacter. It is not the intent to support this type of behavior going forward, due to the significant difficulty and time required to process this form of logic. (As clarity for those not involved in many of the parts of the formula rebuild: Yes, I get some things are conditional. Yes, something needs to be able to conditionally grant. No, you don't need a trailing PRExxx to accomplish those ends. The full explanation of this is handled elsewhere).

=New Storage/Granting=

The Facet system in many cases is providing a simple item or list storage mechanism. The new formula system inherently provides such storage as well, and can do so without separate classes required for each piece of storage. Therefore, the strategic design is moving away from Facets and into storing items in the variable system.

The system will effectively be able to grant certain variables. This will start with Alignment. For the moment, there will be a series of Facets that act as a transition, allowing a pull of certain variables from the new formula system and granting the contents of those variables to the PC.

Full Load Order Detail

2018-03-04T17:17:37Z

Tom Parker:

{| align="right"
| __TOC__
|}

This is a more complete explanation of load processing beyond the simple literal Loader and Token processing. This assumes that the GameModes have been loaded, and that a campaign is undergoing load. That front-end process is described in the [[Rules Persistence System]] and the [[Load Commit Subsystem]].

This deals with the full (and right now necessary) order of operations for loading that occurs after the persistence files are fully loaded from disk. This describes what is happening in pcgen.persistence.SourceFileLoader.
# An internal Ability Object to handle the Bonus Languages is constructed. This is used internally to help provide a foundation for the CHOOSE and other items necessary to support making the Bonus Language choices and granting those choices to the PC.
# Deferred Objects are constructed, if necessary. These are items that are implied by the data, but where we have no specific construct in the data to force their construction (and forcing their construction at every reference would be hazardous). Currently this is limited to CompanionList objects.
# Derived Objects are constructed. This includes many of the skill and spell lists, which are implied by the existance of a PCClass, but for which no specific construct in the data forces their construction. Note that we cannot do these "Live" with the Class construction, because a "KEY:" token which renamed the identifier of the PCClass would mean the identifier of the Derived Objects should be changed as well. So it is currently only safe to derive these after load is complete (and it's actually easier this way).
# We ensure that all Ability Categories that are created are in some way used.
# Deferred Tokens are run. Deferred Tokens are those that intend to process information on an object, but it needs to ensure that all items are loaded before the test or analysis is done. Often this is some form of error checking, such as a FACT token that is REQUIRED:YES for a given FACT ensuring that all objects of a given Format have that FACT defined. Deferred Tokens implement the DeferredToken interface.
# Any unresolved references are identified and UnconstrutedReference errors are thrown, if necessary.
# Any Post-Validation Tokens are run. This includes items that implement PostValidationToken, and are items run across MULTIPLE OBJECTs. These tokens are provided all objects of a given type. This can validate things like "Is there only one SIZE with "DEFAULTSIZE:YES" which are impossible to answer looking one object at a time.
# Any Post-Deferred Tokens are run. These tokens implement the PostDeferredToken interface, and require a fully validated dataset where CDOMReference objects are resolved before they are run. This will do cross-token checks (such as MULT:YES and CHOOSE: travel together), but are only processing one object at a time. As a note, if it is possible, the DeferredToken interface is preferable to the PostDeferredToken interface.
# We validate that the new formula system has a full set of defaults that can be resolved without external dependencies. This has to be done post-resolution since a default (ALIGNMENT is None) might be an object, and we need that reference to have been filled before we test that the default can be properly processed without external dependencies. (Meaning it needs to not have external dependencies AT RUNTIME, it is welcome to have a dependency on a part of the data which will be static at runtime).
# We validate that any reference where a choice was included (e.g. an Exotic Weapon Proficiency FEAT chooses a Weapon Proficiency)... this will detect if someone attempted to grant a CHOOSE including ability without a choice (or provided a choice to an Ability that didn't have a CHOOSE).
# Choices on EquipmentModifiers are processed to ensure they are correctly applied.
# We validate that all weapons have either TYPE:Melee or TYPE:Ranged
# We Auto Generate Equipment (if enabled)
# We initialize any Data Set Facets with required information about the loaded data.

Index to Architecture Documents

2018-02-27T04:38:32Z

Tom Parker: /* Loading */

{| align="right"
| __TOC__
|}

This is effectively a "shorthand" list of documents (and a tiny bit of context around the links) for "A Day in the Life of PCGen"

=Resources=

Before reading about the details of the PCGen architecture, it is likely helpful to understand our key [[Design Concepts for PCGen]] and [[Architecture Document Conventions]]

=Loading=

# The [[Startup System]] initializes PCGen and loads the initial user interface
# Files from Disk are loaded by the [[Rules Persistence System]]
# These files use [[CDOM References Concept Document|CDOM References]] to resolve order of operations conflicts during load and [[Referring to Groups in LST Data|Refer to Groups in LST Data]] to identify multiple possible objects (for say, a choice)
# The [[Load Commit Subsystem]] is used to commit information into the [[Rules Data Store]]
# Post Processing is completed as described in the [[Full Load Order Detail]]

=Using Data=

# When retrieving items from the Rules Data Store, it's important to [[Identifying Objects|Identify the right Object]]

=Processing a PC=

# After [[Adding items to the PC]], we begin [[Calculating Items on the PC]]
# This calculation heavily depends on the [[Formula Systems]]
# Some of the items on a PC can be conditional, so it's valuable to understand [[Prerequisites and Requirements]]

Index to Architecture Documents

2018-02-27T04:37:21Z

Tom Parker:

{| align="right"
| __TOC__
|}

This is effectively a "shorthand" list of documents (and a tiny bit of context around the links) for "A Day in the Life of PCGen"

=Resources=

Before reading about the details of the PCGen architecture, it is likely helpful to understand our key [[Design Concepts for PCGen]] and [[Architecture Document Conventions]]

=Loading=

# The [[Startup System]] initializes PCGen and loads the initial user interface
# Files from Disk are loaded by the [[Rules Persistence System]]
# These files use [[CDOM References Concept Document|CDOM References]] to resolve order of operations conflicts during load
# The files also can [[Referring to Groups in LST Data|Refer to Groups in LST Data]] to identify multiple possible objects (for say, a choice)
# The [[Load Commit Subsystem]] is used to commit information into the [[Rules Data Store]]
# Post Processing is completed as described in the [[Full Load Order Detail]]

=Using Data=

# When retrieving items from the Rules Data Store, it's important to [[Identifying Objects|Identify the right Object]]

=Processing a PC=

# After [[Adding items to the PC]], we begin [[Calculating Items on the PC]]
# This calculation heavily depends on the [[Formula Systems]]
# Some of the items on a PC can be conditional, so it's valuable to understand [[Prerequisites and Requirements]]