This paper presents a measurement of forward-forward and forward-central dijet azimuthal angular correlations and conditional yields in proton-proton (pp) and proton-lead (p + Pb) collisions as a probe of the nuclear gluon density in regions where the fraction of the average momentum per nucleon carried by the parton entering the hard scattering is low. In these regions, gluon saturation can modify the rapidly increasing parton distribution function of the gluon. The analysis utilizes 25 pb(-1) of pp data and 360 mu b(-1) of p + Pb data, both at root S-NN = 5.02 TeV, collected in 2015 and 2016, respectively, with the ATLAS detector at the Large Hadron Collider. The measurement is performed in the center-of-mass frame of the nucleon-nucleon system in the rapidity range between -4.0 and 4.0 using the two highest transverse-momentum jets in each event, with the highest transverse-momentum jet restricted to the forward rapidity range. No significant broadening of azimuthal angular correlations is observed for forward-forward or forward-central dijets in p + Pb compared to pp collisions. For forward-forward jet pairs in the proton-going direction, the ratio of conditional yields in p + Pb collisions to those in pp collisions is suppressed by approximately 20%, with no significant dependence on the transverse momentum of the dijet system. No modification of conditional yields is observed for forward-central dijets.