Not a good test. You really need to observe and make notes of the speeds that FSD was using during your outward drive and then copy that as closely as possible on the return trip when you are driving. This test also assumes that the trip was essentially level for the complete trip (not climbing in one direction and falling in the reverse direction, for example). Temperatures do make a difference but for a short trip of only 40 miles with immediate turn around I wouldn't imagine that would play a significant role. You also said you were going to charge at the remote destination before returning. Did you? If so, then you appear to have a severe problem with your battery maintaining its state of charge.