sqlcmd -S $_DB_CONN -d $_DB -s "|" -h-1 -m -1 -W -i $_SCRIPTDIR/SQL/EXPORT_COMPARE.sql -o $_INPUT/comp_list.txt  set NOCOUNT ON;
select 
a.ABSOLUTE_FILE_PATH     
from DBNAME..TABLE_STG a 
JOIN DBNAME..TABLE_STG b on a.ABSOLUTE_FILE_PATH COLLATE S 
SQL_Latin1_General_CP1_CS_AS = b.ABSOLUTE_FILE_PATH COLLATE 
SQL_Latin1_General_CP1_CS_AS and a.ENV = 'ENV1' and b.ENV = 'ENV2'
LEFT JOIN DBNAME..TABLE_EXCLUSION excl ON a.ENV = excl.ENV AND 
a.ABSOLUTE_FILE_PATH = excl.ABSOLUTE_FILE_PATH
where 
a.ABSOLUTE_FILE_PATH not like '%.rul' and
excl.ABSOLUTE_FILE_PATH is null;
set NOCOUNT OFF;`

sqlcmd -S $_DB_CONN -d $_DB -s "|" -h-1 -m -1 -W -i $_SCRIPTDIR/SQL/EXPORT_COMPARE.sql -o $_INPUT/comp_list.txt  set NOCOUNT ON;

• Edit: The goal of this Question was only seeking code review and guidance in learning or tips for improvement. I apologize if that was not clear or aligned properly goals of this forum.

• Script Goal: audit file existence and differences between two red hat Enterprise Linux Servers 5.11 (Carthage)

• If the files exist in both servers are they the identical?

• Check md5sum: If the md5sum is not a match what is different?

• The file is being copied to a tmp directory on the local server.

• The files are then compared using diff.

• The absolute file path and the environment is added to a diff table in SqlServer.

• The text output of each diff is written to a text file in a control folder and can be displayed in a web app.

• Checks that need to be implemented:

• Do the files contain hard coded server information or passwords?

• If yes add a flag column in the Database.

• Code

   DIR=(/Dir/Durp/DurpaDurp/Scriptdir1 /Dir/Durp/DurpaDurp/Scriptdir2 /Dir/Durp/DurpaDurp/Scriptdir3 ) 
  
   f_Type=("*.sh" "*.txt" "*.log")   
   f_Type2=("*.img" "*.rpt" )
  
   for((i=0; i<${#DIR[@]}; i++)) 
   do
     echo "CHECKING: ${DIR[$i]}"
     cd "${DIR[$i]}" 
         for((x=0; x<${#f_Type[@]}; x++)) 
          do
            echo "FOR FILE TYPE: ${f_Type[$x]}"
           find $PWD -type f -name "${f_Type[$x]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
         done 
   if [[ ${DIR[$i]} == "/Dir/Durp/DurpaDurp/Script3" ]]; 
     then
        for((y=0; y<${#f_Type[@]}; y++)) 
       do
          echo "FOR FILE TYPE: ${f_Type2[$y]}"
         find $PWD -type f -name "${f_Type2[$y]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
       done 
   fi
   done 
  

• After these files are output to the text files the script will delete the existing data from a staging table in SQL Server 2008 R2 and insert the new data.

   for((i=0; i<${#ENV[@]}; i++))
   do
     sqlcmd -S $_DB_CONN -d $_DB  -Q "DELETE FROM ['$_DB']..['$_TABLE'] WHERE ENV = '${ENV[$i]}'";
   done
  
   bcp $_DB_CONN.."$_TABLE" in "$filelistENV1" -f "$_SCRIPTDIR/STG.fmt" -e $_ERRDIR/ERROR_STG$(date -d "today" +"%Y%m%d%H%M").txt -S $_DB_CONN -d "$_DB"
  

• the format file creates 2 columns

  AbsoluteFilePath | ENV

• gets a list of files from the database to compare

• Edit: The goal of this Question was only seeking code review and guidance in learning or tips for improvement. I apologize if that was not clear or aligned properly with the goals of this forum.

• Script Goal: audit file existence and differences between two red hat Enterprise Linux Servers

  • Place information in sql server tables to use for analysis
  • Maintain ability to view the diff in a web page
  • Process Steps
    • Set array with Directorys to check
      • Set array of file types to look for
      • Build a list of existing files for each Environment
  • Script will solve the following
    • If the files exist in both servers are they the identical? (md5)*
    • If no what is different (Diff)?
      • Does the file contain hard coded values (grep)
  • View Diff in web page (currently reading text file using into site php)

• Code

   DIR=(/Dir/Durp/DurpaDurp/Scriptdir1 /Dir/Durp/DurpaDurp/Scriptdir2 /Dir/Durp/DurpaDurp/Scriptdir3 ) 
  
   f_Type=("*.sh" "*.txt" "*.log")   
   f_Type2=("*.img" "*.rpt" )
  
   for((i=0; i<${#DIR[@]}; i++)) 
   do
     echo "CHECKING: ${DIR[$i]}"
     cd "${DIR[$i]}" 
         for((x=0; x<${#f_Type[@]}; x++)) 
          do
            echo "FOR FILE TYPE: ${f_Type[$x]}"
           find $PWD -type f -name "${f_Type[$x]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
         done 
   if [[ ${DIR[$i]} == "/Dir/Durp/DurpaDurp/Script3" ]]; 
     then
        for((y=0; y<${#f_Type[@]}; y++)) 
       do
          echo "FOR FILE TYPE: ${f_Type2[$y]}"
         find $PWD -type f -name "${f_Type2[$y]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
       done 
   fi
   done 
  

• After these files are output to the text files the script will delete the existing data from a staging table in SQL Server 2008 R2 and insert the new data.

   for((i=0; i<${#ENV[@]}; i++))
   do
     sqlcmd -S $_DB_CONN -d $_DB  -Q "DELETE FROM ['$_DB']..['$_TABLE'] WHERE ENV = '${ENV[$i]}'";
   done
  
   bcp $_DB_CONN.."$_TABLE" in "$filelistENV1" -f "$_SCRIPTDIR/STG.fmt" -e $_ERRDIR/ERROR_STG$(date -d "today" +"%Y%m%d%H%M").txt -S $_DB_CONN -d "$_DB"
  

• the format file creates 2 columns

  AbsoluteFilePath | ENV

• gets a list of files from the database to compare

• Goal: audit file existence and differences between two red hat Enterprise Linux Servers 5.11 (Carthage)

• If the files exist in both servers are they the identical?

• Check md5sum: If the md5sum is not a match what is different?

• The file is being copied to a tmp directory on the local server.

• The files are then compared using diff.

• The absolute file path and the environment is added to a diff table in SqlServer.

• The text output of each diff is written to a text file in a control folder and can be displayed in a web app.

• Checks that need to be implemented:

• Do the files contain hard coded server information or passwords?

• If yes add a flag column in the Database.

• Code

   DIR=(/Dir/Durp/DurpaDurp/Scriptdir1 /Dir/Durp/DurpaDurp/Scriptdir2 /Dir/Durp/DurpaDurp/Scriptdir3 ) 
  
   f_Type=("*.sh" "*.txt" "*.log")   
   f_Type2=("*.img" "*.rpt" )
  
   for((i=0; i<${#DIR[@]}; i++)) 
   do
     echo "CHECKING: ${DIR[$i]}"
     cd "${DIR[$i]}" 
         for((x=0; x<${#f_Type[@]}; x++)) 
          do
            echo "FOR FILE TYPE: ${f_Type[$x]}"
           find $PWD -type f -name "${f_Type[$x]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
         done 
   if [[ ${DIR[$i]} == "/Dir/Durp/DurpaDurp/Script3" ]]; 
     then
        for((y=0; y<${#f_Type[@]}; y++)) 
       do
          echo "FOR FILE TYPE: ${f_Type2[$y]}"
         find $PWD -type f -name "${f_Type2[$y]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
       done 
   fi
   done 
  

• After these files are output to the text files the script will delete the existing data from a staging table in SQL Server 2008 R2 and insert the new data.

   for((i=0; i<${#ENV[@]}; i++))
   do
     sqlcmd -S $_DB_CONN -d $_DB  -Q "DELETE FROM ['$_DB']..['$_TABLE'] WHERE ENV = '${ENV[$i]}'";
   done
  
   bcp $_DB_CONN.."$_TABLE" in "$filelistENV1" -f "$_SCRIPTDIR/STG.fmt" -e $_ERRDIR/ERROR_STG$(date -d "today" +"%Y%m%d%H%M").txt -S $_DB_CONN -d "$_DB"
  

• the format file creates 2 columns

  AbsoluteFilePath | ENV

• gets a list of files from the database to compare

• Edit: The goal of this Question was only seeking code review and guidance in learning or tips for improvement. I apologize if that was not clear or aligned properly goals of this forum.

• Script Goal: audit file existence and differences between two red hat Enterprise Linux Servers 5.11 (Carthage)

• If the files exist in both servers are they the identical?

• Check md5sum: If the md5sum is not a match what is different?

• The file is being copied to a tmp directory on the local server.

• The files are then compared using diff.

• The absolute file path and the environment is added to a diff table in SqlServer.

• The text output of each diff is written to a text file in a control folder and can be displayed in a web app.

• Checks that need to be implemented:

• Do the files contain hard coded server information or passwords?

• If yes add a flag column in the Database.

• Code

   DIR=(/Dir/Durp/DurpaDurp/Scriptdir1 /Dir/Durp/DurpaDurp/Scriptdir2 /Dir/Durp/DurpaDurp/Scriptdir3 ) 
  
   f_Type=("*.sh" "*.txt" "*.log")   
   f_Type2=("*.img" "*.rpt" )
  
   for((i=0; i<${#DIR[@]}; i++)) 
   do
     echo "CHECKING: ${DIR[$i]}"
     cd "${DIR[$i]}" 
         for((x=0; x<${#f_Type[@]}; x++)) 
          do
            echo "FOR FILE TYPE: ${f_Type[$x]}"
           find $PWD -type f -name "${f_Type[$x]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
         done 
   if [[ ${DIR[$i]} == "/Dir/Durp/DurpaDurp/Script3" ]]; 
     then
        for((y=0; y<${#f_Type[@]}; y++)) 
       do
          echo "FOR FILE TYPE: ${f_Type2[$y]}"
         find $PWD -type f -name "${f_Type2[$y]}" | sed 's/^/ENV1|/'| column -t >> "$filelistENV1"
       done 
   fi
   done 
  

• After these files are output to the text files the script will delete the existing data from a staging table in SQL Server 2008 R2 and insert the new data.

   for((i=0; i<${#ENV[@]}; i++))
   do
     sqlcmd -S $_DB_CONN -d $_DB  -Q "DELETE FROM ['$_DB']..['$_TABLE'] WHERE ENV = '${ENV[$i]}'";
   done
  
   bcp $_DB_CONN.."$_TABLE" in "$filelistENV1" -f "$_SCRIPTDIR/STG.fmt" -e $_ERRDIR/ERROR_STG$(date -d "today" +"%Y%m%d%H%M").txt -S $_DB_CONN -d "$_DB"
  

• the format file creates 2 columns

  AbsoluteFilePath | ENV

• gets a list of files from the database to compare