Primeramente debes poner el control picturebox en el formulario le das clic derecho sobre el picturebox y seleccionas la opcion "Enviar al fondo" y debe de funcionar y colocas tus objetos sobre el picturebox:
Dim oldX As Long, oldY As Long, isMoving As Boolean
Private Sub Command1_MouseDown(Button As Integer, Shift _
As Integer, X As Single, Y As Single)
oldX = X
oldY = Y
isMoving = True
End Sub
Private Sub Command1_MouseMove(Button As Integer, _
Shift As Integer, X As Single, Y As Single)
If isMoving Then
Command1.Top = Command1.Top - (oldY - Y)
Command1.Left = Command1.Left - (oldX - X)
End If
End Sub
Private Sub Command1_MouseUp(Button As Integer, _
Shift As Integer, X As Single, Y As Single)
isMoving = False
End Sub
Nota.Si no te funciona te envío el ejemplo a tu correo.
Saludos!!!