Handling data with pandas
++++++++++++++++++++++++++++++++++++++++

.. _labpandas:

Overview
=====================================================
We can consulte the slides about pandas in the following link


In this folder you will find some laboratories about database managment with pandas (a python library) the purpose not is 
present notebooks to deploy rather to expose some methods or functions.


Dataframe structure
----------------------------------------------------------
The main object with interact with data.




Laboratories
-------------------------------------------------------------
The objective is that you can find how solve some
problems according to keywords


Load data
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. `load url (video) <https://www.youtube.com/watch?v=yzdhyzRmAB4&t=6s>`_
#. `load from local (video) <https://www.youtube.com/watch?v=yzdhyzRmAB4&t=6s>`_


Change name and drop columns
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. `Change name of columns colab <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/change_name_columns(ai_page).ipynb>`_

#. `Create and drop columns <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Restaurants(ai_page)sol.ipynb>`_



Counting Missing values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. `Nans colab <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Nans(ai_page).ipynb>`_



Filter values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. `Filter lab I <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/clase_filtro_pandas(BIT).ipynb>`_
#. `Filter lab II <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/labPandas3(BIT).ipynb>`_



Groupby
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. `Bankrupt by sector <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/bankrupt_sector(groupby).ipynb>`_
#. `Introduction to groupby  <https://github.com/it-ces/PUBLIC-AI/blob/main/PANDAS(talleres-slides)/Labs/GroupBy(ai_page).ipynb>`_
#. `Excersice using groupby <https://github.com/it-ces/PUBLIC-AI/blob/main/PANDAS(talleres-slides)/Labs/lab-share-market.ipynb>`_


Dates
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. `Intro dates <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/dates_(AI_PAGE).ipynb>`_
#. `Count transactions <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Count_transactions(ai_page).ipynb>`_



Duplicates values 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#.  `Lab duplicated <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/BIT_lab_duplicates.ipynb>`_


#. `Lab duplicated groupby <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Duplicates_groupby(BIT).ipynb>`_




Summation or mean over columns or rows
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Sometimes you need calculate the mean or sum a set of columns or the values of a variable

#.   `Lab mean-sum <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/DataFrame_sum_mean(BIT).ipynb>`_



Replace
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. `Replace values <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Pandas_to_replace(BIT).ipynb>`_



Merge and append dataframes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. `Basic merge <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/clase_merge(BIT).ipynb>`_

#. `Loop-merge <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Loop_dataframes(BIT).ipynb>`_

#. `Variables consistency <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Check_variables_loop(BIT).ipynb>`_


Imputation values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#. `Basic imputation <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Nulls_imputation(BIT).ipynb>`_


Practical laboratories
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The following laboratory is to review some concepts with practical results, to realize 
this lab dowloand the database `pibs-dptos.xlsx` avaliable `(Click here) <https://github.com/it-ces/Datasets/blob/main/pibs-dptos.xlsx>`_

#. `Health lab <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/LaboratoryPandas(ONE).ipynb>`_
#. `Gdps lab I <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Pandas_manejo(BIT).ipynb>`_
#. `Gdps lab II <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Lab_PIBS(BIT).ipynb>`_
#. `World bank data <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/solow_data(ai_page).ipynb>`_
#. `TMR lab <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/TMR_model(ai_page).ipynb>`_
#. `Hanushek lab <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Hanushek_hypothesis(Ai_page).ipynb>`_

Regular expressions (Regex)
------------------------------------------------------------------------------------------

This topic deserve "apartado" given its importance.
"Al principio" regex could be messy, but with practice each time its implementation
is more inutitive.


Regex lab
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

`regex basic <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/freq_problems_pandas(BIT).ipynb>`_



Laboratories I
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* `Contigency Table <https://github.com/it-ces/labHealth/blob/master/Health_Second_Class.ipynb>`_



Reshaping data
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

*   `Reshape data <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/reshape_data_pivot(ai_page).ipynb>`_
*   `Pivot data <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/pivot_pivotTable(ai_page).ipynb>`_

Table one
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
#. `Table one bankrupt <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/Tableone(ai-page).ipynb>`_



Visualization 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
* `Introduction to matplolib <https://github.com/it-ces/PUBLIC-AI/blob/952e12ad3b9f65ee11717c2974be8c4cf7613bc0/short-labs-pandas/introduction%20to%20matplotlib.ipynb#L10>`_


Hands-on Pandas workshop 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* `Taller 1 <https://github.com/it-ces/Talleres/blob/main/tallerU1.pdf>`_
* `Taller 2 <https://github.com/it-ces/Talleres/blob/main/TallerU2.pdf>`_
* `Taller 3 <https://github.com/it-ces/Talleres/blob/main/TallerU3.pdf>`_
* `Taller 4 <https://github.com/it-ces/Talleres/blob/main/work_overdue.pdf>`_
* `Taller 5 <https://github.com/it-ces/Talleres/blob/main/taller-clothing-store.pdf>`_
* `Taller 6 <https://github.com/it-ces/Talleres/blob/main/taller-sales-semester.pdf>`_



Solutions Pandas workshop 
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

* `Taller 1 <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/taller1_sol(ai_pageU).ipynb>`_
* `Taller 2 <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/taller2_sol(ai_pageU).ipynb>`_
* `Taller 3 <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/taller3_sol(ai_pageU).ipynb>`_
* `Taller 4 <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/taller4_sol(ai_pageU).ipynb>`_
* `Taller 5 <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/taller5_sol(ai_pageU).ipynb>`_
* `Taller 6 <https://github.com/it-ces/PUBLIC-AI/blob/main/short-labs-pandas/taller6_sol(ai_pageU).ipynb>`_