Using {targets}, {arrow}, Docker, Postgres and the Command Line for Medium Data by Jared P. Lander
Visit [ Ссылка ] to learn more.
Abstract: Most companies don't have big data, but rather medium data, that awkward in between where the data are too big to fit in memory but not big enough for Google-scale systems. Fortunately R has many options for working with data of this size. We will look at using the command line, {data.table} and {dplyr} to clean the data and load it into a Postgres database inside a Docker container. Then we will use {targets} to orchestrate the whole process.
Bio: Jared P. Lander is Chief Data Scientist of Lander Analytics, the Organizer of the New York Open Statistical Programming Meetup and the New York R Conference and an Adjunct Professor of Statistics at Columbia University. With a masters from Columbia University in statistics and a bachelors from Muhlenberg College in mathematics, he has experience in both academic research and industry. Jared oversees the long-term direction of the company and acts as Lead Data Scientist, researching the best strategy, models and algorithms for modern data needs. This is in addition to his client-facing consulting and training. He specializes in data management, multilevel models, machine learning, generalized linear models, data management, visualization and statistical computing. He is the author of R for Everyone, the best-selling book about R Programming geared toward Data Scientists and Non-Statisticians alike. The book is available from Amazon, Barnes & Noble and InformIT. The material is drawn from the classes he teaches at Columbia and is incorporated into his corporate training. Very active in the data community, Jared is a frequent speaker at conferences, universities and meetups around the world. His writings on statistics can be found at jaredlander.com.
Twitter: [ Ссылка ]
Presented at the 2021 Government & Public Sector R Conference (December 9, 2021)
