Add Generation IX Shape Data

[KyroChi]

Closes #1120

I have added the generation IX shape data (I think).

I have never contributed to PokéAPI, please tell me anything I am doing here that is stupid or out of line.

The PokéAPI's shape naming differs from Bulbapedia, please double check that I got the conversion correct.

Here is the automated script I used to scrape the data:

import csv
import re
import requests

from bs4 import BeautifulSoup
from pages import load_homepage
from tqdm.auto import tqdm

data_out = {}

BASE_URL = 'https://bulbapedia.bulbagarden.net'
LANDING = '/wiki/List_of_Pok%C3%A9mon_by_National_Pok%C3%A9dex_number'

pokedata = load_homepage(BASE_URL + LANDING)

for ii, pokemon in (pb := tqdm(enumerate(pokedata[905:]))):
    pb.set_postfix_str(f"Processing {pokemon['name']}")

    url = BASE_URL + pokemon['link']
    response = requests.get(url)
    
    if response.status_code == 200:
        soup = BeautifulSoup(response.text, 'html.parser')

        pattern = re.compile(r'File:Body\d{2}.png')
        matches = soup.find_all('a', href=pattern)

        body_type = matches[0]['href'].split('.')[-2][-2:]

        data_out[ii + 1] = int(body_type)
    else:
        raise Exception(f"Failed to get data from {url}, status code: {response.status_code}")
    

# json.dump(data_out, open('/home/kyle/projects/pokemon_data/src/scrape/gen_ix_shapes.json', 'w'))
# data_out = json.load(open('/home/kyle/projects/pokemon_data/src/scrape/gen_ix_shapes.json'))

csv_file = '/home/kyle/code/pokeapi/pokeapi/data/v2/csv/pokemon_species.csv'

bulbapedia_to_pokeapi = {
    1: 1,
    2: 5,
    3: 2,
    4: 6,
    5: 8,
    6: 10,
    7: 7,
    8: 4,
    9: 9,
    10: 13,
    11: 3,
    12: 12,
    13: 14,
    14: 11,
}

with open(csv_file, 'r') as f:
    species_data = list(csv.DictReader(f))

for row in species_data:
    if int(row['id']) < 906:
        continue
    else:
        row['shape_id'] = str(bulbapedia_to_pokeapi[data_out[str(int(row['id']) - 905)]])

with open(csv_file, 'w') as f:
    f.write(','.join(species_data[0].keys()) + '\n')
    for row in species_data:
        f.write(','.join(row.values()) + '\n')

[KyroChi]

@Naramsim I am happy to make modifications to this PR. To be perfectly honest I am hoping my simple solution here solved the problem, but I don't necessarily have the time to dive in more deeply.

[Naramsim]

Hi, I can't review it now, but it seems very promising!

I'll update you in the next days hopefully

[programgames]

what is the library that you use : 'pages' , I didn't found it on internet

[KyroChi]

what is the library that you use : 'pages' , I didn't found it on internet

Aw, shoot. That is some code from another library that I have been working on. I have updated the original PR to include this without reference to my own module. Apologies.

[pokeapi-machine-user]

A PokeAPI/api-data refresh has started. In ~45 minutes the staging branch of PokeAPI/api-data will be pushed with the new generated data.

The staging branch will be deployed in our staging environment and the entire API will be ready to review.

A Pull Request (master<-staging) will be also created at PokeAPI/api-data and assigned to the PokeAPI Core team to be reviewed. If approved and merged new data will soon be available worldwide at pokeapi.co.

[pokeapi-machine-user]

The updater script has finished its job and has now opened a Pull Request towards PokeAPI/api-data with the updated data.

The Pull Request can be seen deployed in our staging environment when CircleCI deploy will be finished (check the start time of the last build).