Python Day 65 – importing my own data

Today I tried taking some data and messing around with it. I was able to import the .csv but didn’t get to manipulating it. I learned: While this was mostly reviewing old notes/workbooks of mine, I did learn that there is a big conceptual difference between a database and dataframe. I’m not 100% certain butContinue reading “Python Day 65 – importing my own data”

Python Day 64 – practicing numpy and thoughts on the future

I’m done with the API module so I just went back to do some numpy practice problems to see if I even still remembered it haha. I learned: Just the little stuff, like when using np.arrange to use (parenthesis) and not [brackets], but when I’m making adjustments to an array to use [brackets]. Looking forward:Continue reading “Python Day 64 – practicing numpy and thoughts on the future”

Python Day 63 – more data pulling

Today I continued with the website data module, which is starting to make more sense. I learned: There are two main ways to look at data from websites in a more elegant manner, manually or with ‘glom’. For instance, to get posts titles on reddit, one can use: titles = []For post in data[‘data’][‘children’]: titles.append(post[‘data’][‘title’])titlesContinue reading “Python Day 63 – more data pulling”

Python Day 62 – pulling data from websites

Today I started a module on HTTP requests and APIs, which is essentially about pulling data from websites. I learned: The library for HTTP requests is ‘requests’, and one of the most useful functions here is the .get request. Once one .get’s a website, one can check if the connection is successful with the .status_codeContinue reading “Python Day 62 – pulling data from websites”

Python Day 61 – shifting and rolling data

Today I finished the module on time series! I learned: One can shift data over by a certain time amount with .shift(). One can use ‘rolling windows’ to average out data over a period of time. This can be done with past/future data (by setting center = True), but is usually done with just pastContinue reading “Python Day 61 – shifting and rolling data”

Python Day 60 – index manipulation of timestamp

As the title says, I went over some manipulations of data that can be done once the timestamp is set as the index. I learned: To set the timestamp as the index with: data.Timestamp = pd.to_datetime(data.Timestamp, infer_datetime_format=True)data = data.set_index(‘Timestamp’, drop=True)  And once we have the timestamp as the index, we can use the .loc functionContinue reading “Python Day 60 – index manipulation of timestamp”

Python Day 59 – one step forward two steps back in datetime

I realized for my problem from yesterday with a custom datetime, if I changed ‘%-m’ to ‘%m’, the only other error was it having trouble reading the second value in the time position. So I threw in a cheeky ‘%H:00’ which is definitely wrong but it worked! And I got no errors. However… I amContinue reading “Python Day 59 – one step forward two steps back in datetime”

Python Day 58 – more datetime stuffs

I figured out why it wasn’t letting me subtract dates before! It was because one value was a date (year, month, day) and the other value was a date AND time (including hours, mins, seconds). The difference was really between using .datetime and .date. So I just made them both be .datetime and it worked!Continue reading “Python Day 58 – more datetime stuffs”

Python Day 57 – timestamps

Today I continued with the data wrangling and learned about timestamps. I learned: Timestamps are the most basic form of time series data for Pandas. Apparently they’re used in almost all data (usually counting when the data event happened). Their class is a string, however you can convert it to datetime format (so we canContinue reading “Python Day 57 – timestamps”

Python Day 56 – regex and datetime

Today I did some more regex crosswords. Still couldn’t figure out the one that stumped me yesterday, although I managed to do a few more until I reached this one which is currently stumping me: So then I moved on into this worksheet for wrangling time series data… I learned: To import datetime. The basicContinue reading “Python Day 56 – regex and datetime”