Slice notation allows you to quickly take a subset of a vector or matrix, by using a short expression in place of the indices.

In MATLAB you may be familiar with

>> a = [1,2,3,4,5,6,7,8,9,10]; >> a(3:7) ans = 3 4 5 6 7

The “slice notation” a(3:7) takes the subset of the vector a starting at index 3, up to and including index 7.

Slice notation in Python is similar, but *just different enough* to cause endless confusion for MATLAB programmers.

In this post we’ll start with the worst of it. In Python, the starting index of any list is 0, and the last value in the slice is *not included* in the indices. The best way to see how this screws things up is by example:

MATLAB | Python |

>> a = [1,2,3,4,5]; >> a(1:4) ans = 1 2 3 4 >> a(4) ans = 4 |
>>> a = [1,2,3,4,5] >>> a[1:4] [2, 3, 4] >>> a[4] 5 |

So what happened here?

In MATLAB it’s clear: a(1:4) is the vector consisting of a(1), a(2), a(3), and a(4), from the first index to the fourth — nice and intuitive.

In Python the starting value is strictly the same, a[1] — but because vector indices start at 0, a[1] is the *second* element of the list: a[1] = 2. For this you can probably blame the C language, which uses this convention.

So you might expect a[1:4] to consist of a[1], a[2], a[3], a[4]. But no — the last entry in the slice is *excluded* from the list for some reason. Thus we have a[1:4] consisting of a[1], a[2], and a[3] (which are 2, 3, and 4, respectively). In the above example, the fifth element of the list, a[4] = 5, is outside the slice.

I find this to be bafflingly counterintuitive, and it took a long time before I got comfortable with it. But the good news is Python slice notation has useful features which are not found in MATLAB; I’ll explore those in a future post.

]]>As you know, we recently embarked on an exciting renovation to expand the Grand Hotel. If you have stayed with us before, you may have noticed that your room numbers are now represented by real numbers, rather than just integers. Many guests have remarked how much they like the new numbering system.

However, we regret that additional renovations are necessary, and some rooms will be temporarily closed. During this time, we will need to move all guests to integer-numbered rooms.

To thank you for your patience, the infinity pool will be open extended hours.

*D. Hilbert, Manager, Grand Hotel*

To: Manager <dhilbert@grandhotel.com>

From: Assistant Manager <gcantor@grandhotel.com>

Subject: Are you sure?

David — We’re pretty full in the reals. I’ve been running the numbers and I’m not sure we’ll be able to fit all our guests in the integers. Can you let me know the plan.

Regards, Georg

To: Assistant Manager <gcantor@grandhotel.com>

From: Manager <dhilbert@grandhotel.com>

Subject: Re: Are you sure?

Georg, don’t worry — we never run out of rooms here. Remember when we had that termite infestation in the odd-numbered rooms? We just took the guest in room n and put them in 2n.

I mean, we even had to do this once when the reno was half done and we had guests in all the rational-numbered rooms. Not a problem then either.

David

To: Manager <dhilbert@grandhotel.com>

From: Assistant Manager <gcantor@grandhotel.com>

Subject: Re: Are you sure?

It would help me out a lot if you can write down what you are thinking.

To: Assistant Manager <gcantor@grandhotel.com>

From: Manager <dhilbert@grandhotel.com>

Subject: Re: Are you sure?

Georg, you know you’ll never get promoted if you don’t show initiative? But fine. Here’s what we’ll do.

First we’ll write down a list of all the real-valued room numbers, in any order you like.

Then we’ll enumerate the list, assigning an integer to each one: the first room in the list gets number 1, the second gets number 2, and so on down the list.

Now each real-valued room has an integer. Then we just give each guest the number of their room on the list. Simple as pie.

To: Manager <dhilbert@grandhotel.com>

From: Assistant Manager <gcantor@grandhotel.com>

Subject: URGENT — Call me

Hi David, it’s 3 AM and I’ve been calling and calling but there’s no answer. We have a big problem.

You remember the guest who checked in last night – Di Agonal? She had a strange request about her room.

I was making the list of rooms like you asked – Di looked over my shoulder and said, “I want a room with a unique number. Take the first digit of the first room on your list, and add 1; that will be the first digit in my room number. Then the 2nd digit of the 2nd room on the list, again adding 1; that will be the second digit. And so on.”

“What if the digit is 9?” I asked. “Then just turn it into a zero,” she said.

I didn’t think much about it — Di’s requested room was a real number just like all the others, and it happened to be available. I gave her the key and sent her on her way with the bellhop.

But when we went to move the guests into integer-numbered rooms, *Di’s room number wasn’t on the list*. The room number wasn’t the first element on the list, since the first digit didn’t match. And it wasn’t the second, since the second digit didn’t match. And so on. But my list had all the integers. *Can it be that we have more rooms than integers?*

I don’t mind telling you that Di is livid. And now more angry guests are coming to see me because they lost their rooms in the move. What do we do?

*Memo to All Guests*

Here at the Grand Hotel, we have always prided ourselves on having rooms available no matter who comes to our door.

Last night we let you down. Fortunately our friends at the Hotel Zermelo-Fraenkel were able to accommodate all of you.

Our former manager has taken full responsibility for last night’s events and has resigned. He asked me to convey his uncountable apologies to you.

As our way of making it up to you, complimentary prime rib will be served in the restaurant at 2, 3, 5, 7, and 11.

*G. Cantor, Manager, Grand Hotel*

*Scene: A classics department faculty lounge; a few professors are relaxing. *

Prof 1 [rushes into the room]: Guys! Guys! You remember that cache of magnetic recordings they found in the ruins of old Silicon Valley?

Prof 2: What about it?

P1 [catching her breath]: Most of it was corrupt, but we finally decoded one. It’s a short moving picture.

P2: We already know about the mindless entertainments of the ancients.

P1: No, don’t you see – this one is in two languages. The audio track is in ancient German, but there are written words on top of the images – and I’m positive they’re in English.

P2: A discovery like this would be the biggest since … well, what are you waiting for, show us!

*P1 presses a hidden button. A holographic screen appears in the middle of the room and a movie begins to play:*

Here’s some example code (remember this is best done in Anaconda):

import matplotlib.pyplot as plt import numpy as np x = np.arange(-5,5,0.01) fx = (1/np.sqrt(2*np.pi))*np.exp(-1*np.power(x,2)/2) fx2 = (1/np.sqrt(4*np.pi))*np.exp(-1*np.power(x,2)/4) fx_tex = '$f(x)=\\frac{1}{\\sqrt{2\\pi}}\\exp\\left(-\\frac{x^2}{2}\\right)$' fx2_tex = '$f(x)=\\frac{1}{\\sqrt{4\\pi}}\\exp\\left(-\\frac{x^2}{4}\\right)$' plt.figure(1) plt.plot(x,fx,label=fx_tex) plt.plot(x,fx2,label=fx2_tex) plt.xlabel('$x$') plt.ylabel('$f(x)$') plt.legend() plt.show()

Notice how the legend labels are specified by the “label” tag in the plot command. Also, remember to escape those backslashes!

And here’s the output:

]]>In fact *q*(*x*) is a posterior distribution for an “intensity” channel. Suppose we have a binary channel output $$y \in \{0,1\},$$ with *x* as the channel input. Further suppose, given *x*, we have $$\mathrm{Pr}(y = 1 \:|\: x_i) = k x_i$$ for some constant *k*; that is, the probability of turning *y* “on” is proportional to the input *x*. Then $$\mathrm{Pr}(y = 1) = \sum_i p(x_i) k x_i = k E[X]$$ and $$\Pr(x_i \:|\: y = 1) = \frac{p(x_i) k x_i}{k E[X]} = q(x_i).$$

I call this an “intensity” channel because the intensity of the input *x* is the key parameter in the input-output relation. A physical example would be a light sensor, where *x* represents the intensity of the light source (i.e., density of photons) and *y* = 1 represents the arrival of a photon at the sensor.

I feel like distributions of the form of *q*(*x*) must have a name and must be well studied. Drop me a line if you know more.

Here’s how it works: for a list n with elements n[0], n[1], …, n[N-1], pick an index uniformly distributed over 0,1,…,N-1. That index is swapped with n[0]. (If you pick index 0, the swap is trivial.) Then an index over 1,2,…,N-1 is swapped with n[1]; over 2,3,…,N-1 is swapped with n[2], and so on through the list.

Note the shuffle is recursive: after swapping n[i], the operation over n[i+1],…,n[N-1] is a Knuth shuffle.

You can show by induction that all N! possible permutations can be generated by this algorithm. For N=2, the first step leaves the list the same, or swaps n[0] and n[1]; the second step is trivial, so there are 2 = 2! possible permutations. Since the algorithm is recursive, assume the shuffle over n[1],…,n[N-1] produces (N-1)! possible permutations. There are N possible swaps of n[0] with an index in 0,1,…,N-1; thus, there are a total of N(N-1)! = N! permutations. A similar inductive argument shows that the shuffle is uniformly distributed over the permutations.

Here’s a Knuth shuffle in Python, with a few demonstration lines at the end.

import random as rnd def shuffle(n,start=0): # base case: # the only remaining element is the last one if (start == len(n)-2): return n # recursive case: # swap n[start] with a randomly selected element # from the rest of the list swapIndex = rnd.randint(start,len(n)-1) foo = n[start] n[start] = n[swapIndex] n[swapIndex] = foo return shuffle(n,start+1) # demonstration n = [1,2,3,4,5,6,7,8,9,10,11,12,13,14,15] print(n) print(shuffle(n))

]]>

Over the next few weeks I’ll give some tips for MATLAB users who are migrating to Anaconda. For now, I’ll assume you are familiar with the basics of Python 3 syntax (which is not too hard, particularly if you know MATLAB; see here to get started).

[Updating] List of tips:

]]>

Using the GET command, I typed “GET /~aeckford/helloworld.html” and retrieved that page from my department’s server:

(Also try: telnet towel.blinkenlights.nl )

]]>$$\mathrm{tr}(A) = \sum_{i=1}^n A_{ii} .$$

This might not seem too exciting at first. However, the trace operator has a neat quasi-commutative property: for matrices *U* and *V*, so long as the internal dimensions work out, it is true that

$$\mathrm{tr}(UV) = \mathrm{tr}(VU) .$$

The proof isn’t too hard so I’ll skip it. If we had a third matrix *W* (again assuming the internal dimensions work out), since matrix multiplication is associative, it is also true that

$$\mathrm{tr}(UVW) = \mathrm{tr}(WUV) = \mathrm{tr}(VWU) .$$

It’s not truly commutative, since you can only do cyclic shifts of the arguments. So, e.g., tr(*UVW*) is not equal to tr(*WVU*) in general.

What can you do with this? For one thing, note that the trace of a scalar a is itself: tr(*a*) = *a*. So if you have a matrix multiplication that results in a scalar, you can use trace to rearrange the arguments.

For instance, let *U* be a 1 by *n* row vector, and let *V* be an *n* by *n* matrix. If *U** is the transpose of *U*, then *UVU** is a scalar. This kind of expression comes up pretty often in jointly Gaussian distributions.

Now say U is a zero-mean vector with covariance matrix E[*U***U*], and I want to know E[*UVU**]. Using the trace trick, I can express this expectation in terms of E[*U***U*]: first, we can write

$$UVU^* = \mathrm{tr}(UVU^*) = \mathrm{tr}(VU^*U) ,$$

and since expectation distributes over the trace sum, we have

$$E[\mathrm{tr}(VU^*U)] = \mathrm{tr}(E[VU^*U]) = \mathrm{tr}(V E[U^*U]) .$$

As a result, if you know the covariance E[*U*U*], there’s no need to recalculate any expectations.

*This is a repost: original is at http://andreweckford.blogspot.ca/2009/09/trace-tricks.html*

For now, this will be a tutorial-style blog. Posts will consist of quick tips, tricks, and lessons on subjects that I (and hopefully you) find useful.

]]>