The use of semi-transparent photovoltaic devices causes an inevitable loss of photocurrent, as light transmitted has not been absorbed. This trivial effect also leads to a loss of photovoltage, an effect partially due to the lower photocurrent but also due to the geometry of the semi-transparent photovoltaic device. We here demonstrate and evaluate this photovoltage loss in semi-transparent organic photovoltaic devices, compared with non-transparent solar cells of the same material. Semi-transparent solar cells in addition introduce photovoltage loss when formed by lamination. We document and analyze these effects for a number of polymer blends in the form of bulk heterojunctions.