A member of the House of Capet, Philip was born in the medieval fortress of Fontainebleau (Seine-et-Marne) to the future Philip III, the Bold, and his first wife, Isabella of Aragon. He was the second of four sons born to the couple. His father was the heir apparent of France at that time, being the eldest son of King Louis IX (better known as St. Louis).
In August 1270, when Philip was two years old, his grandfather died while on Crusade, his father became king, and his elder brother Louis became heir apparent. Only five months later, in January 1271, Philip's mother died after falling from a horse; she was pregnant with her fifth child at the time and had not yet been crowned queen beside her husband. A few months later, one of Philip's younger brothers, Robert, also died. Philip's father was finally crowned king at Rheims on 15 August 1271. Six days later, he married again; Philip's step-mother was Marie, daughter of the duke of Brabant.
In May 1276, Philip's elder brother Louis died, and the eight year old Philip became heir apparent. It was suspected that Louis had been poisoned, and that his stepmother, Marie of Brabant, had instigated the murder. One reason for these rumours was the fact that the queen had given birth to her own first son the month Louis died. However, both Philip and his surviving full brother Charles lived well into adulthood and raised large families of their own.
The scholastic part of Philip's education was entrusted to Guillaume d'Ercuis, his father's almoner.
After the unsuccessful Aragonese Crusade against Peter III of Aragon, which ended in October 1285, Philip may have negotiated an agreement with Peter for the safe withdrawal of the Crusader army. This pact is attested to by Catalan chroniclers. Joseph Strayer points out that such a deal was probably unnecessary, as Peter had little to gain from provoking a battle with the withdrawing French or angering the young Philip, who had friendly relations with Aragon through his mother.
Philip married Queen Joan I of Navarre (1271–1305) on 16 August 1284. The two were affectionate and devoted to each other and Philip refused to remarry after Joan's death in 1305, despite the great political and financial rewards of doing so. The primary administrative benefit of the marriage was Joan's inheritance of Champagne and Brie, which were adjacent to the royal demesne in Ile-de-France, and thus effectively were united to the king's own lands, expanding his realm. The annexation of wealthy Champagne increased the royal revenues considerably, removed the autonomy of a large semi-independent fief and expanded royal territory eastward. Philip also gained Lyon for France in 1312.
Navarre remained in personal union with France, beginning in 1284 under Philip and Joan, for 44 years. The Kingdom of Navarre in the Pyrenees was poor but had a degree of strategic importance. When in 1328 the Capetian line went extinct, the new Valois king, Philip VI, attempted to permanently annex the lands to France, compensating the lawful claimant, Joan II of Navarre, senior heir of Philip IV, with lands elsewhere in France. However, pressure from Joan II's family led to Phillip VI surrendering the land to Joan in 1329, and the rulers of Navarre and France were again different individuals.