We can verify our results by zooming in on the Nyquist plot for different gains using the zoom command.

K = 0.79

sys=tf([1 10 24], [1 -8 15]);
w=logspace(-2,3,501);
k=0.79;
nyquist(k*sys,w);

Note that the Nyquist diagram does not loop around the -1 point.

We will now look at our step response. Remember that we must look at the closed-loop step response.

sys_cl = feedback(k*sys,1);
step(sys_cl)

The system is unstable.

K = 0.80

k=0.80;
nyquist(k*sys,w);

Note that the Nyquist diagram goes right through the -1 point.

We will now look at our step response. Remember that we must look at the closed-loop step response.

sys_cl = feedback(k*sys,1);
step(sys_cl)

The system is marginally stable.

K = 0.81

k=0.81;
nyquist(k*sys,w)

Note that the Nyquist diagram encircles the -1 point.

We will now look at our step response. Remember that we must look at the closed-loop step response.

sys_cl = feedback(k*sys,1);
step(sys_cl)

The system is stable.