We report about the properties of the underlying event measured with ALICE at the LHC in pp and p$-$Pb collisions at $\sqrt{s_{\rm NN}}=5.02$ TeV. The event activity, quantified by charged-particle number and summed-$p_{\rm T}$ densities, is measured as a function of the leading-particle transverse momentum ($p_{\rm T}^{\rm trig}$). These quantities are studied in three azimuthal-angle regions relative to the leading particle in the event: toward, away, and transverse. Results are presented for three different $p_{\rm T}$ thresholds (0.15, 0.5, and 1 GeV/$c$) at mid-pseudorapidity ($|\eta|<0.8$). The event activity in the transverse region, which is the most sensitive to the underlying event, exhibits similar behaviour in both pp and p$-$Pb collisions, namely, a steep increase with $p_{\rm T}^{\rm trig}$ for low $p_{\rm T}^{\rm trig}$, followed by a saturation at $p_{\rm T}^{\rm trig} \approx 5$ GeV/$c$. The results from pp collisions are compared with existing measurements at other centre-of-mass energies. The quantities in the toward and away regions are also analyzed after the subtraction of the contribution measured in the transverse region. The remaining jet-like particle densities are consistent in pp and p$-$Pb collisions for $p_{\rm T}^{\rm trig}>10$ GeV/$c$, whereas for lower $p_{\rm T}^{\rm trig}$ values the event activity is slightly higher in p$-$Pb than in pp collisions. The measurements are compared with predictions from the PYTHIA 8 and EPOS LHC Monte Carlo event generators.