Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions scrape_term.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,15 +69,19 @@ def parse(content, db):
rows = t.find_all('tr', {'class': 'CourseRow'})
s = defaultdict(lambda: defaultdict(list))
for tr in rows:
cols = tr.find_all(lambda tag: tag.name == 'td' and not tag.get_text().isspace())
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🏳️ 🏳️ 🏳️

cols = tr.find_all(lambda tag: tag.name == 'td')

if cols:
# The first <td> is a field that is not relevant to us
# it is either empty or contains a "flag" icon
cols.pop(0)
Comment thread
phi-line marked this conversation as resolved.

for i, c in enumerate(cols):
a = c.find('a')
cols[i] = (a.get_text() if a else cols[i].get_text()).strip()

try:
key = get_key(f'{cols[0] if cols[0] else cols[1]}')[0]
key = get_key(cols[0])[0]
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh wait, not sure if this is necessary / what it breaks.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is okay now because you popped the first.
Can you test locally to see if it returns a desired result?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to my testing, it seems to work fine. (Tested with LiveMyPortalData)

data = dict(zip(HEADERS, cols))

crn = data['CRN']
Expand Down