Joseph had been taken to Egypt by the Ishmaelites, who had sold him to Potiphar, an Egyptian who was one of Pharaoh's officers, the commander of the royal guard.
They were just sitting down to have a meal when they saw a caravan of Ishmaelites coming from Gilead. Their camels were carrying aromatic spices, balm, and myrrh to take to Egypt.
So when the Ishmaelites (who were traders from Midian) came by, they pulled Joseph out of the pit and sold him to them for twenty pieces of silver. The Ishmaelites took him to Egypt.
In the meantime the Ishmaelites had arrived in Egypt and had sold Joseph to Potiphar. Potiphar was one of Pharaoh's officers, the captain of the guard.