Dynamic package.xml generation

Some weeks after implementing Continuous Integration as described in my previous post (Continuous integration for Salesforce), we started to see that the build was taking longer and longer. We compared different deployment logs and we saw that even do we were adding some new stuff, we were also deploying all the existing components as well including those that were not changed. So we started thinking and researching how to make our package.xml based on what we changed, and we came across this post: Dynamically Building Package.xml

Using this as a base we came with the following bash script:

#/usr/bin/env bash
# -lcommit builds last commit
# -prevrsa last commit to master

#read command line args
while getopts l:p: option
do
        case "${option}"
        in
                l) LCOMMIT=${OPTARG};;
                p) PREVRSA=${OPTARG};;
        esac
done

echo Last Commit: $LCOMMIT
echo Previous Commit: $PREVRSA

DIRDEPLOY=build/deploy
if [ -d "$DIRDEPLOY" ]; then
    echo Removing deploy folder
    rm -rf "$DIRDEPLOY"
fi
mkdir -p $DIRDEPLOY
cd src
echo changing directoy to src
cp package.xml{,.bak} &&
echo Backing up package.xml to package.xml.bak

NEWPKGXML=$(<packageBase.xml) echo $NEWPKGXML > package.xml
echo List of changes
echo DIFF: `git diff-tree --no-commit-id --name-only --diff-filter=ACMRTUXB -t -r $PREVRSA $LCOMMIT`

git diff-tree --no-commit-id --name-only --diff-filter=ACMRTUXB -t -r $PREVRSA $LCOMMIT | \
while read -r CFILE; do

        if [[ $CFILE == *"src/"*"."* ]]
        then
                tar cf - "../$CFILE"* | (cd ../$DIRDEPLOY; tar xf -)
        fi
        if [[ $CFILE == *"-meta.xml" ]]
        then
                ADDFILE=$CFILE
                ADDFILE="${ADDFILE%-meta.xml*}"
                tar cf - ../$ADDFILE | (cd ../$DIRDEPLOY; tar xf -)
        fi
        if [[ $CFILE == *"/aura/"*"."* ]]
        then
                DIR=$(dirname "$CFILE")
                tar cf - ../$DIR | (cd ../$DIRDEPLOY; tar xf -)
        fi

        case "$CFILE"
        in
                *.snapshot*) TYPENAME="AnalyticSnapshot";;
                *.cls*) TYPENAME="ApexClass";;
                *.component*) TYPENAME="ApexComponent";;
                *.page*) TYPENAME="ApexPage";;
                *.trigger*) TYPENAME="ApexTrigger";;
                *.approvalProcess*) TYPENAME="ApprovalProcess";;
                *.assignmentRules*) TYPENAME="AssignmentRules";;
                */aura/*) TYPENAME="AuraDefinitionBundle";;
                *.autoResponseRules*) TYPENAME="AutoResponseRules";;
                *.community*) TYPENAME="Community";;
                */applications*.app*) TYPENAME="CustomApplication";;
                *.customApplicationComponent*) TYPENAME="CustomApplicationComponent";;
                *.labels*) TYPENAME="CustomLabels";;
                *.md*) TYPENAME="CustomMetadata";;
                */objects/*__*__c.object*)
                    TYPENAME="UNKNOWN TYPE" # We don't want objects from managed packages to be deployed;;
                */objects*.object*) TYPENAME="CustomObject";;
                *.objectTranslation*) TYPENAME="CustomObjectTranslation";;
                *.weblink*) TYPENAME="CustomPageWebLink";;
                *.customPermission*) TYPENAME="CustomPermission";;
                *.tab*) TYPENAME="CustomTab";;
                */documents/*.*) TYPENAME="Document";;
                *.email*) TYPENAME="EmailTemplate";;
                */email/*-meta.xml) TYPENAME="EmailTemplate";;
                *.escalationRules*) TYPENAME="EscalationRules";;
                *.globalValueSet*) TYPENAME="GlobalValueSet";;
                *.globalValueSetTranslation*) TYPENAME="GlobalValueSetTranslation";;
                *.group*) TYPENAME="Group";;
                *.homePageComponent*) TYPENAME="HomePageComponent";;
                *.homePageLayout*) TYPENAME="HomePageLayout";;
                *.layout*) TYPENAME="Layout";;
                *.letter*) TYPENAME="Letterhead";;
                *.permissionset*) TYPENAME="PermissionSet";;
                *.cachePartition*) TYPENAME="PlatformCachePartition";;
                *.profile*) TYPENAME="Profile";;
                *.reportType*) TYPENAME="ReportType";;
                *.role*) TYPENAME="Role";;
                *OrgPreference.settings*) TYPENAME="UNKNOWN TYPE";;
                *.settings*) TYPENAME="Settings";;
                */standardValueSets*.standardValueSet*) TYPENAME="StandardValueSet";;
                *.standardValueSetTranslation*) TYPENAME="StandardValueSetTranslation";;
                *.resource*) TYPENAME="StaticResource";;
                *.translation*) TYPENAME="Translations";;
                *.workflow*) TYPENAME="Workflow";;
                *) TYPENAME="UNKNOWN TYPE";;
        esac

        if [[ "$TYPENAME" != "UNKNOWN TYPE" ]]
        then

                case "$CFILE"
                in
                        src/email/*)  ENTITY="${CFILE#src/email/}";;
                        src/documents/*)  ENTITY="${CFILE#src/documents/}";;
                        src/aura/*)  ENTITY="${CFILE#src/aura/}" ENTITY="${ENTITY%/*}";;
                        *) ENTITY=$(basename "$CFILE");;
                esac

                if [[ $ENTITY == *"-meta.xml" ]]
                then
                        ENTITY="${ENTITY%%.*}"
                        ENTITY="${ENTITY%-meta*}"
                else
                        ENTITY="${ENTITY%.*}"
                fi

                if grep -Fq "<name>$TYPENAME</name>" package.xml
                then
                        xmlstarlet ed -L -s "/Package/types[name='$TYPENAME']" -t elem -n members -v "$ENTITY" package.xml
                else
                        xmlstarlet ed -L -s /Package -t elem -n types -v "" package.xml
                        xmlstarlet ed -L -s '/Package/types[not(*)]' -t elem -n name -v "$TYPENAME" package.xml
                        xmlstarlet ed -L -s "/Package/types[name='$TYPENAME']" -t elem -n members -v "$ENTITY" package.xml
                fi
        fi
done

echo Cleaning up Package.xml
xmlstarlet ed -L -i /Package -t attr -n xmlns -v "http://soap.sforce.com/2006/04/metadata" package.xml

echo ====FINAL PACKAGE.XML=====
cat package.xml
tar cf - package.xml | (cd ../$DIRDEPLOY/src; tar xf -)

Let’s start analyzing what this script is doing.

The following lines take the arguments from the command line and assigns to variables. -l is for the last commit and -p is the base commit to compare against.

#read command line args
while getopts l:p: option
do
case "${option}"
in
l) LCOMMIT=${OPTARG};;
p) PREVRSA=${OPTARG};;
esac
done

Then we check if the folder where we are going to move what we are going to deploy exists (from previous runs). If so we delete it and create it again:

DIRDEPLOY=build/deploy
if [ -d "$DIRDEPLOY" ]; then
    echo Removing deploy folder
    rm -rf "$DIRDEPLOY"
fi
mkdir -p $DIRDEPLOY

The key command here is the one that retrieves the changes from git between the commits that we sent as arguments:

echo DIFF: `git diff-tree --no-commit-id --name-only --diff-filter=ACMRTUXB -t -r $PREVRSA $LCOMMIT`

Another important section of the code if what comes below. In here we analyze what we retrieved from git and start adding the components and component types to the package.xml file as well as copy those files to the specific folder  that will be later zipped:


while read -r CFILE; do

        if [[ $CFILE == *"src/"*"."* ]]
        then
                tar cf - "../$CFILE"* | (cd ../$DIRDEPLOY; tar xf -)
        fi
        if [[ $CFILE == *"-meta.xml" ]]
        then
                ADDFILE=$CFILE
                ADDFILE="${ADDFILE%-meta.xml*}"
                tar cf - ../$ADDFILE | (cd ../$DIRDEPLOY; tar xf -)
        fi
        if [[ $CFILE == *"/aura/"*"."* ]]
        then
                DIR=$(dirname "$CFILE")
                tar cf - ../$DIR | (cd ../$DIRDEPLOY; tar xf -)
        fi

        case "$CFILE"
        in
                *.snapshot*) TYPENAME="AnalyticSnapshot";;
                *.cls*) TYPENAME="ApexClass";;
                *.component*) TYPENAME="ApexComponent";;
                *.page*) TYPENAME="ApexPage";;
                *.trigger*) TYPENAME="ApexTrigger";;
                *.approvalProcess*) TYPENAME="ApprovalProcess";;
                *.assignmentRules*) TYPENAME="AssignmentRules";;
                */aura/*) TYPENAME="AuraDefinitionBundle";;
                *.autoResponseRules*) TYPENAME="AutoResponseRules";;
                *.community*) TYPENAME="Community";;
                */applications*.app*) TYPENAME="CustomApplication";;
                *.customApplicationComponent*) TYPENAME="CustomApplicationComponent";;
                *.labels*) TYPENAME="CustomLabels";;
                *.md*) TYPENAME="CustomMetadata";;
                */objects/*__*__c.object*)
                    TYPENAME="UNKNOWN TYPE" # We don't want objects from managed packages to be deployed
                    ;;
                */objects*.object*) TYPENAME="CustomObject";;
                *.objectTranslation*) TYPENAME="CustomObjectTranslation";;
                *.weblink*) TYPENAME="CustomPageWebLink";;
                *.customPermission*) TYPENAME="CustomPermission";;
                *.tab*) TYPENAME="CustomTab";;
                */documents/*.*) TYPENAME="Document";;
                *.email*) TYPENAME="EmailTemplate";;
                */email/*-meta.xml) TYPENAME="EmailTemplate";;
                *.escalationRules*) TYPENAME="EscalationRules";;
                *.globalValueSet*) TYPENAME="GlobalValueSet";;
                *.globalValueSetTranslation*) TYPENAME="GlobalValueSetTranslation";;
                *.group*) TYPENAME="Group";;
                *.homePageComponent*) TYPENAME="HomePageComponent";;
                *.homePageLayout*) TYPENAME="HomePageLayout";;
                *.layout*) TYPENAME="Layout";;
                *.letter*) TYPENAME="Letterhead";;
                *.permissionset*) TYPENAME="PermissionSet";;
                *.cachePartition*) TYPENAME="PlatformCachePartition";;
                *.profile*) TYPENAME="Profile";;
                *.reportType*) TYPENAME="ReportType";;
                *.role*) TYPENAME="Role";;
                *OrgPreference.settings*) TYPENAME="UNKNOWN TYPE";;
                *.settings*) TYPENAME="Settings";;
                */standardValueSets*.standardValueSet*) TYPENAME="StandardValueSet";;
                *.standardValueSetTranslation*) TYPENAME="StandardValueSetTranslation";;
                *.resource*) TYPENAME="StaticResource";;
                *.translation*) TYPENAME="Translations";;
                *.workflow*) TYPENAME="Workflow";;
                *) TYPENAME="UNKNOWN TYPE";;
        esac

        if [[ "$TYPENAME" != "UNKNOWN TYPE" ]]
        then

                case "$CFILE"
                in
                        src/email/*)  ENTITY="${CFILE#src/email/}";;
                        src/documents/*)  ENTITY="${CFILE#src/documents/}";;
                        src/aura/*)  ENTITY="${CFILE#src/aura/}" ENTITY="${ENTITY%/*}";;
                        *) ENTITY=$(basename "$CFILE");;
                esac

                if [[ $ENTITY == *"-meta.xml" ]]
                then
                        ENTITY="${ENTITY%%.*}"
                        ENTITY="${ENTITY%-meta*}"
                else
                        ENTITY="${ENTITY%.*}"
                fi

                if grep -Fq "<name>$TYPENAME</name>" package.xml
                then
                        xmlstarlet ed -L -s "/Package/types[name='$TYPENAME']" -t elem -n members -v "$ENTITY" package.xml
                else
                        xmlstarlet ed -L -s /Package -t elem -n types -v "" package.xml
                        xmlstarlet ed -L -s '/Package/types[not(*)]' -t elem -n name -v "$TYPENAME" package.xml
                        xmlstarlet ed -L -s "/Package/types[name='$TYPENAME']" -t elem -n members -v "$ENTITY" package.xml
                fi
        fi
done

As you can see, we have to do some manual checks. For example, in the following line we check if what has been changed is the “-meta.xml”  file, we want to also copy the related component file (the one that doesn’t end with -meta.xml). If we don’t do this and try to deploy only an xml file, Salesforce will throw an error:

if [[ $CFILE == *"-meta.xml" ]]
        then
                ADDFILE=$CFILE
                ADDFILE="${ADDFILE%-meta.xml*}"
                tar cf - ../$ADDFILE | (cd ../$DIRDEPLOY; tar xf -)
        fi

If you are using lightning components, another important logic to have is to move the complete aura folder, even do only one of the files was changed:

if [[ $CFILE == *"/aura/"*"."* ]]
        then
                DIR=$(dirname "$CFILE")
                tar cf - ../$DIR | (cd ../$DIRDEPLOY; tar xf -)
        fi

As you can see this section will changed based on what metadata you are keeping in your repository. You may have more or less than what I put here, but the idea should be the same.

As a final step we print the resulting package.xml file and move it to the specified folder:

echo ====FINAL PACKAGE.XML=====
cat package.xml
tar cf - package.xml | (cd ../$DIRDEPLOY/src; tar xf -)

Now, how do you run all this. Well, this will depend on the OS that you are using. As you can see, this is a bash script, so you will need a Unix like OS or an emulator.

For Linux:

1- Install xmlstarlet

2- Install git

3- Run the command: bash build/dynamicBash.sh -l {lastCommit} -p {lastCommitToMaster}

For Windows:
1- You first need to install a Linux virtual machine. I highly recommend to install CygWin (https://www.cygwin.com/). When installing it also add the following packages: dos2Unix and xmlstarlet.
2- You will also need Git for Windows: https://git-scm.com/download/win
3- Open Cygwin and navigate to your repository folder
4- Run the command: dos2unix.exe build/dynamicBash.sh
5- Run the command: bash build/dynamicBash.sh -l {lastCommit} -p {lastCommitToMaster}

For Mac OS:

1- Install gnu-tar using brew

2- You will now need to update the PATH environment variable so Terminal defaults to using gnutar instead of bsdtar

3- Navigate to your repository folder

4- Run the following command: $ ./build/dynamicBash.sh -l {lastCommit} -p {lastCommitToMaster} 

In any of the OS that you use, the command is the same where lastCommit is the latest commit in your branch and lastCommitToMaster is the starting one, meaning the one that was successfully deployed previously. This should result on the creation of a new folder inside the build folder (build\deploy\src) in where you will see the new package.xml.

As you can see, the script can be improved, but it works. I recommend that you try it and let me know your results. Obviously, if you do any enhancement, please share it, and I will gladly add a later post with those changes 🙂

If you are already doing CI as explained in my previous post, you can include this script making the deployments to be smaller and faster.

Advertisements

One thought on “Dynamic package.xml generation

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s