CSV files with Python

1. Pandas library

   • pip install pandas
     

   ---

   • Read CSV file

     import pandas as pd
     pd.read_csv('data.csv')
           

   ---

   • Get number of rows and columns

     data_frame = pd.read_csv('data.csv')
     data_frame.shape # (number of rows, number of columns) Example: (18207, 89)
     

   ---

   
   
   • Get summary statistics: count, min, max, ...

       data_frame.describe()
     

   ---

   
   
   • Get all values in the data frame

       data_frame.values
     

   
   

   ---

   • Filter: get first 5 rows with age > 40

     data_frame[data_frame["Age"]>40].head()
     

   
   

   ---

   • Get selected columns

       pd.DataFrame(data_frame, columns=['Name', 'Wage', 'Value']) 
     

     In Jupyter Notebook:

       df1 = pd.DataFrame(data_frame, columns=['Name', 'Wage', 'Value'])
       df1
     

   
   

   ---

   • Calculate, manipulate data and add a column. Sort a column.

     def value_to_float(x):
         if type(x) == float or type(x) == int:
             return x
         if 'K' in x:
             if len(x) > 1:
                 return float(x.replace('K', '')) * 1000
             return 1000.0
         if 'M' in x:
             if len(x) > 1:
                 return float(x.replace('M', '')) * 1000000
             return 1000000.0
         if 'B' in x:
             return float(x.replace('B', '')) * 1000000000
         return 0.0
     

     ---

     df1 = pd.DataFrame(data_frame, columns=['Name', 'Wage', 'Value'])  
     value =  df1['Value'].replace('[\€,]', '', regex=True).apply(value_to_float)
     wage =  df1['Wage'].replace('[\€,]', '', regex=True).apply(value_to_float)
     
     df1['Wage'] = wage
     df1['Value'] = value
     df1['difference'] = df1['Value'] - df1['Wage']
     df1.sort_values('difference', ascending=False) # Used in Jupyter Notebook: 
     

     How to convert the string 2.90K to 2900 or 5.2M to 5200000 in pandas dataframe.



---

2. CSV library

   ...

---

3. Additional Resources

   • Python Cheat Sheet for Data Science
   
   
   • Pandas dataframe example