Looping over list of dicts and removing elements

Question

I'm facing a problem that I don't really understand. Please take a look at the given code, where the two relevant variables have the following form:

jsd = {"data": {"User": .....}}
lists = [{"genre": "XY", "meanScore": Float},..., {"genre": "XY", "meanScore": Float}]

The code is:

jsd = json.loads(r.text)  
lists = jsd["data"]["User"]["stats"]["favouredGenresOverview"]
for k, entry in enumerate(lists):
    if entry["meanScore"] == None:
        del(lists[k])

The goal is to remove all the dict's in lists where the key meanScore is equal to None. The problem is that for some reason the loop only runs over half of lists... I think I've read somewhere on the site that when trying to remove something from a dict one should use pop, but I honestly wasn't able to figure out how to that or if it even was relevant to my question (As far as I understand I'm removing list elements).

Can someone maybe help?

Don't delete items from a list while iterating; plenty of duplicates for the strange behaviour but I think that one best illustrates the skipping of elements. Build a new list of things you want to _keep_ rather than delete elements. — roganjosh, Dec 19 '18 at 22:17
@ParitoshSingh don't modify the _length_ of what you are iterating through. Maybe I have a list of integers and I want to multiply them all by 2; I wouldn't have this issue. — roganjosh, Dec 19 '18 at 22:20
Instead of removing unwanted items from the original list, add wanted items to a new list. As others have said, don't iterate and remove from a collection at the same time, if you do so you desserve all kinds of nasty things, and you're getting them. If you need na explanation of why and what is happening, let me know. — Pedro Rodrigues, Dec 19 '18 at 22:22
@PedroRodrigues that is already covered fully in the duplicate I have flagged — roganjosh, Dec 19 '18 at 22:25
@roganjosh Yep, stackoverflow was outdated on my side. Thumbs up — Pedro Rodrigues, Dec 20 '18 at 09:23

score 2 · Accepted Answer · answered Dec 19 '18 at 22:26

A more pythonic approach to these type of problems is to think of what you want to "keep" instead.

jsd = json.loads(r.text)  
lists = jsd["data"]["User"]["stats"]["favouredGenresOverview"]

output_list = [entry for entry in lists if entry["meanScore"] is not None]

You can get fancier with a generator and slicing if you want to do it inplace.

lists[:] = (entry for entry in lists if entry["meanScore"] is not None)

Remy J · Answer 2 · 2018-12-19T22:44:13.273

0

lists = [{"genre": "XY", "meanScore": 'Float'}, {"genre": "XY", "meanScore": None}]
[d for d in lists if d['meanScore'] is not None]

This will give the following output

[{'meanScore': None, 'genre': 'XY'}]

edited Dec 19 '18 at 22:44

answered Dec 19 '18 at 22:24

Remy J

709
1
7
18

1

Though `None` is falsy, so if much else! To include only `None`, an explicit `if d['meanScore'] is not None` is needed. – jmd_dk Dec 19 '18 at 22:27
1

Heads up though, that condition would falsify empty strings/0/empty lists and so on. – Paritosh Singh Dec 19 '18 at 22:27
@jmd_dk You're correct. I'll fix that. – Remy J Dec 19 '18 at 22:29

Looping over list of dicts and removing elements

2 Answers2