Answer the following questions on information theory. Suppose that we transmit information by using a time-discrete communication channel , whose input and output are designated as and , respectively.
The input and output relation of the -th communication via is represented as , where means the multiplication of integers.
is an internal state of the channel at the -th communication, and its value can change depending on the current or past states of the input and on the past states of the output.
Both sender and receiver are unable to observe the value of directly although they can have knowledge about how changes depending on the input and output. Use the logarithm base for your answers of the following questions.
You may also use the following approximations upon necessity: , , .
(1) Let be an ideally independent random variable that takes with probability and with probability . Assume that becomes with probability when and that it takes either or with equal probability when .
(1-i) Obtain the entropies and and the conditional entropy of .
(1-ii) Obtain the channel capacity of .
(2) Assume that takes either or with equal probability and that, for , the value of becomes the same as the previous output value with probability as .Obtain the maximum bits that can be transmitted by using this channel times.
(3) Assume that takes either or with equal probability when is odd and that keeps its previous value with probability as when is even.Obtain the channel capacity of and show a code that can achieve the capacity.
(4) Assume that with probability and that, for , the value of becomes the same as the previous input value with probability as . Let be an ideally independent random variable that takes with probability and with probability .Calculate the probability that at the stationary state for sufficiently large .
Answer the following questions on signal processing. Consider the two infinite impulse response systems shown in Figs. and . and are the input and output signal sequences of system in Fig. , respectively, and represent the signal values at time for .Similarly, and are the input and output sequences of system in Fig. . The circuits consist of adders, coefficient multipliers, and delays, whose respective functions are described in Fig. .
(1) Obtain the impulse response of system , , and its -transform .
(2) Calculate the frequency response of system 1 and explain the filtering function of this system on the input signal.
(3) Obtain the parameter values of and that makes system equivalent to system .
(4) Draw an equivalent circuit of system that has a smaller number of delays than the original system shown in Fig. .
