Programming in GIS - Data Manipulation with Python

                              

The output for Lab 4 tied together several different components that help view, organize, and edit data that can be opened in Arc programs and in Python. 

In Lab 4 we created a script that would create a new database, copy files into it from a different folder, then iterate over some of those files to find specific features using a cursor. 

The incredible usefulness of cursors in python and data management is immediately apparent, as it allows the user to automate searches and changes to large datasets that might otherwise take quite a bit of time. For what we did in the lab, it's also not necessary to open these datafiles within ArcGIS in order to view the information the cursor retrieves and edit or organize it into a new place. As a professional sometimes working on systems with slow and buggy connections to a remote GIS Desktop, I deeply appreciate the ability to manipulate GIS data files with just python. 

Some areas where I had to struggle with this script were also the places where I learned the most -the basic search cursor and dictionary syntax were available on ESRI reference material, so the lessons came largely in implementing SQL expressions and calling the right parts of the list for the printed output. 

I had originally been trying to avoid using field delimiter, because I was fairly certain I already understood how I needed to format the quotation marks in the where_clause to preserve the string format, and also make the SQL expression to select a specific subset of features. 

This was proven to be incorrect. 

After several rounds of creating variables to contain my SQL expression and tweaking quotations and parenthesis -only to receive alternating errors thrown by the where_clause and the SQL expression- I finally just added and called my column of features with arcpy.AddFieldDelimiters, then listed it in the where_clause with my SQL expression. Having it written this way immediately stopped the str and sql errors, and I will make a point of just using AddFieldDelimiters next time. 

For the most part I felt like this assignment went smoothly; even though I set it down several times to come back to it with the SQL expression. I feel like I've gotten confident enough that I know when something is wrong because I'm uncertain of the syntax, and when something is wrong but should be working, so it's likely just a typo or software issue. 

An outline flowchart of the processes used in the script for this lab. I think I will get in the habit of sketching these out beforehand for long multi-process pieces of code. 

Lessons Learned:

• arcpy.AddFieldDelimiters is a tool for a reason, use it!
• It's best practice to keep your for loops close to the functions they're calling from, less change you'll type something that changes the outcome in between
• Mind your quotation marks! " and ' need to track properly though their blocks
• \ tells python that what comes after will get special treatment, in SQL this needs to come before each of the ' string quotations 
• When in doubt, close ArcPro and rerun the script
• Search, Insert, and Update cursors are highly valuable tools for big datasets, make sure you check number of items before printing though
• When copying files in python, use the basename describe tool to avoid extension and filetype errors
• the \n is inside the quotes, or after an argument with a  +