Issue

This Content is from Stack Overflow. Question asked by Agung

I’d like to gather some lecturer info and export it into csv. I read some article and tutorial using python. The code looks like this

from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
import pandas as pd
from bs4 import BeautifulSoup

driver = webdriver.Chrome(ChromeDriverManager().install())

url = "https://uin-suka.ac.id/id/page/detil_dosen/197606110000002301"

driver.get(url)

soup = BeautifulSoup(driver.page_source, 'lxml')

df = pd.read_html(str(soup))[0]
print(df)

and the output look like this

                             0  1                                   2
0                         Nama  :     Dr. Nina Mariani Noor, SS., MA.
1                Program Studi  :   Interdisciplinary Islamic Studies
2                     Fakultas  :                        Pascasarjana
3       Jenis Pegawai | Status  :  Pegawai Tetap BLU | Aktif Mengajar
4  Jabatan Akademik | Golongan  :                      Lektor | III/C
5                        Email  :                                   -
6          Pendidikan Terakhir  :                                  S3

Process finished with exit code 0

the csv table i created look like this

table

the problem is, how can i pull the second column and put it into csv as row?
thankyou

Solution

without using df and assuming input is dict

keys = res.keys()
with open("output.csv", "w", newline="") as f:
    dict_writer = csv.DictWriter(f, keys)
    dict_writer.writeheader()
    dict_writer.writerows(res)

using df I think you need to transpose it first.

df = df.drop('1',axis=1)
df = df.set_index('col1').T
df.to_csv('output.csv')

This Question was asked in StackOverflow by Agung and Answered by Solly It is licensed under the terms of CC BY-SA 2.5. - CC BY-SA 3.0. - CC BY-SA 4.0.