Edit: I thought your formulation was correct but not really:

We flip the coin after adding, but we also flip the coin even if we didn't add the word (because it was already there). This is subtle!

wrong:

    if k not in mem:
        mem += [k]
        if np.random.rand() > p:
            mem.remove(k)

    if k not in mem:
        mem += [k]
    else:
        if np.random.rand() > p:
            mem.remove(k)

    if k not in mem:
        mem += [k]
    if k in mem:      # not the same than "else" here
        if np.random.rand() > p:
            mem.remove(k)

    if k not in mem:
        mem += [k]
    if np.random.rand() > p:
        mem.remove(k)