4 File Management

4.1 The Essentials

1. Set up a file structure that looks like the following:

Figure 4.1: The basic structure for analysis keeps everything organized. This structure can be reused for almost any analysis

You can use whichever tool you like. Windows Explorer (or equivalent) is probably be the easiest.

2. Start a R.project in the R analysis folder - File > New Project > Existing Directory

3. Load the here package:

if(!require(here)) install.packages("here")
#> Loading required package: here
#> here() starts at D:/Analysing-Cytometry-Data-With-R

4. Check that the here package “starts” at the right location - it will start where the R Project was set up.

If you need to modify the working directory you can do so with the set_here() function (see below for further details)

5. Download/Copy the tutorial fcs files from here.

6. Copy the downloaded fcs files into the fcs folder and the metadata files into the other folder in your newly created file structure.

4.2 A Deeper Dive

One of the biggest advantages of analysing your data using R is setting up workflows and structures that can be easily reused regardless of the data. This begins with how you set up your file structure and how you move around within that file structure to achieve tasks. During the analysis we will be creating and modifying files and figures so we need to make sure that we are saving them in the correctly and in the correct place.

4.3 Navigating the file structure in R

getwd()
#> [1] "D:/Analysing-Cytometry-Data-With-R"

tells us the current directory that any r functions will work upon. The wd is short for working directory. As you can see currently we are on the D: drive working in a folder called “Analysing-Cytometry-Data-With-R”, which is where this tutorial is being written.

Note that for file structures in R we have to use a / instead of a \ we would normally use in Windows for instance. This is because \ is an “escape” character, whcih is ignored by R.

We can explore what folders (or directories as R likes to call them) are in this directory by using:

setwd("/R_Analysis_001")
list.dirs()
#> [1] "."            "./Data"       "./Data/fcs"  
#> [4] "./Data/other" "./Images"     "./Outputs"

We can also look at the files present in the working directory with

list.files()
#>  [1] "_book"                                
#>  [2] "_bookdown.yml"                        
#>  [3] "_bookdown_files"                      
#>  [4] "_common.R"                            
#>  [5] "_output.yml"                          
#>  [6] "01-Introduction.md"                   
#>  [7] "01-Introduction.Rmd"                  
#>  [8] "02-cross-refs_files"                  
#>  [9] "02-R-Environment-setup.md"            
#> [10] "02-R-Environment-setup.Rmd"           
#> [11] "03-Packages.md"                       
#> [12] "03-Packages.Rmd"                      
#> [13] "04-File-Management.Rmd"               
#> [14] "05-Data-Import.Rmd"                   
#> [15] "06-Gating-Data.Rmd"                   
#> [16] "07-Cleaning-Data.Rmd"                 
#> [17] "Analysing-Cytometry-Data-with-R"      
#> [18] "Analysing-Cytometry-Data-With-R.Rproj"
#> [19] "book.bib"                             
#> [20] "chicago-fullnote-bibliography.csl"    
#> [21] "Data"                                 
#> [22] "Draft Chapters"                       
#> [23] "Helios3-1_2022-01-11"                 
#> [24] "Images"                               
#> [25] "index.md"                             
#> [26] "index.Rmd"                            
#> [27] "packages.bib"                         
#> [28] "preamble.tex"                         
#> [29] "R_Analysis_001"                       
#> [30] "README.md"                            
#> [31] "render1d9c321930b0.rds"               
#> [32] "render21443d606b19.rds"               
#> [33] "Rtest.txt"                            
#> [34] "Scripts"                              
#> [35] "style.css"                            
#> [36] "Test.rds"

As you can see, this lists the files and the directories btu if you have a lot of files and folders in your main directory it can be difficult to understand a long mixed list like this.

If we want to change the current working directory we can use

setwd("..") # move up in the directory
getwd() # check if it worked
#> [1] "D:/"
setwd("/Analysing-Cytometry-Data-With-R/R_Analysis_001") # We can use relative paths - note the ommitted D:
getwd() # check once more we;re back where we need to be.
#> [1] "D:/Analysing-Cytometry-Data-With-R/R_Analysis_001"
setwd("~") # go to home directort

4.4 The here package

File and fodler structure is an important part of doing an organised analysis. Different packages require different versions of the file path iAs you can see, we need to keep a careful eye on which directory we are working in, while also knowing the full path to each folder and file. This creates a problem. The pathway is specific to the computer you’re working on. This makes it difficult to share code as each person has to change the file paths to their specific needs.

4.5 Set up a folder structure for data and results

We can create folders with:

dir.create(here("Data"))
#> Warning in dir.create(here("Data")):
#> 'D:\Analysing-Cytometry-Data-With-R\Data' already exists

WE can create files with

here()
file.create(here("Rtest.txt"))

#> Warning in dir.create(here("R_Analysis_001")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001' already
#> exists
#> Warning in dir.create(here("R_Analysis_001/Data")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Data'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Outputs")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Outputs'
#> already exists
#> Warning in
#> dir.create(here("R_Analysis_001/Outputs/flowCut")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Outputs\flowCut'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Images")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Images'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Images/png")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Images\png'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Images/pdf")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Images\pdf'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Data/fcs")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Data\fcs'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Data/Other")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Data\Other'
#> already exists
#> Warning in dir.create(here("R_Analysis_001/Data/RDS")):
#> 'D:\Analysing-Cytometry-Data-With-R\R_Analysis_001\Data\RDS'
#> already exists

4.6 Copy fcs and panel files to the new folder structure