Using Emacs - 23 - Org Capture 1

Org mode has an amazing feature - org-capture. I only started using it recently but it's really awesome.

What took me so long?

Probably that the manual seemed a little intimidating.

It really isn't hard, though. Combined with what I'll go over in the next video and post (and a future one for Google calendar sync) it's what I use for:

  • recording blog ideas
  • managing bookmarks
  • taking short Notes
  • managing my calendar.

I use the binding:

(global-set-key (kbd "C-c c")

By default, running org-capture brings up a form to capture a "task" but things get really cool when you start making your own capture templates. With capture templates, you can speed up recording information and then tell org-mode to store it where you want it.

Here are my template settings.

(setq org-capture-templates
      '(("a" "Appointment" entry (file  "~/Dropbox/orgfiles/" "Appointments")
	 "* TODO %?\n:PROPERTIES:\n\n:END:\nDEADLINE: %^T \n %i\n")
	("n" "Note" entry (file+headline "~/Dropbox/orgfiles/" "Notes")
	 "* Note %?\n%T")
	("l" "Link" entry (file+headline "~/Dropbox/orgfiles/" "Links")
	 "* %? %^L %^g \n%T" :prepend t)
	("b" "Blog idea" entry (file+headline "~/Dropbox/orgfiles/" "Blog Topics:")
	 "* %?\n%T" :prepend t)
	("t" "To Do Item" entry (file+headline "~/Dropbox/orgfiles/" "To Do Items")
	 "* %?\n%T" :prepend t)
	("j" "Journal" entry (file+datetree "~/Dropbox/")
	 "* %?\nEntered on %U\n  %i\n  %a")
	("s" "Screencast" entry (file "~/Dropbox/orgfiles/")
	 "* %?\n%i\n")))

Let's look at the Blog entry:

("b" "Blog idea" entry (file+headline "~/Dropbox/orgfiles/" "Blog Topics:")
 "* %?\n%T" :prepend t)

It's a list with a bunch of options. The first string b is the key to type to select this template, Blog idea is the actual name for this template.

The next item, entry describes the type of template you're creating. entry says you're going to create an org node with a headline which can then be added to an org-file. You can also have your template create things like an org list item a checkbox checkitem or even plain text. You can see the full set of options here.

Next, in parentheses, you have the target - where to store the captured info. The Blog idea template says file+headline followed by a filename in quotes and a headline in quotes. This says to store the new org heading you're capturing in the file "~/Dropbox/orgfiles/" under the heading "Blog Topics:" Contrast this to the appointments template which just specifies file and a filename. The appointments template will just append the new appointment to its specified file whereas Blog idea will prepend (due to the :prepend option at the end of the template) to the "Blog Topics:" heading in

After this we have the actual template: "* %?\n%T" - the %? and %T are placeholders for substitutions. The %T prompts for a date with a timestamp and the %? is where to leave the cursor for final input before saving the capture.

Finally, :prepend and other options can be found at the above link.

All the substitutions can be found in the docs.

Once you've set up your templates, it's quick and easy to capture information.

This video shows the basics. The next one shows how to pop up a capture even when you're not in emacs and how I use org-capture to maintain my web bookmarks.

More information on capture templates can be found here.

Inverted Index Project

I haven't spoken much about the class I've been teaching this semester. It's an intro CS course - a programming heavy intro. I decided to use Python with a transition at the end to C++. The transition is to mirror Hunter's normal first CS course that ends with a C++ intro to prepare the students for next semester's CS course which is a more intense OOP class using C++ - the language we use in our core courses.

Throughout the semester I've tried to use a variety of interesting application areas so as to try to give the students some idea of the possibilities that studying CS will open up for them.

After covering Python dictionaries and lists I thought we'd play by building an inverted Index.

The basic idea is to map a set of words back to source files. For example, given the following four one line files:

files     contents
file.01     if you prick us do we not bleed
file.02     if you tickle us do we not laugh
file.03     if you poison us do we not die and
file.04     if you wrong us shall we not revenge

You could build a data structure mapping each word back to the file(s) that contain it (partially shown here),

Word   Files containing It's
if   file.01 file.02 file.03 file.04
you   file.01 file.02 file.03 file.04
prick   file.01
us   file.01 file.02 file.03 file.04
do   file.01 file.02 file.03

You can, of course, store more information - how many times a word appears in a file, where it appears, etc.

This is a fairly easy structure to build. A dictionary where the keys are the words in the file and the values are lists of the documents containing the words.

  inverted_index = {
      'if' : ['file.01','file.02','file.03','file.04'],
      'you' : ['file.01','file.02','file.03','file.04'],
      'prick' : ['file.01'],
      'us' : ['file.01','file.02','file.03','file.04'],
      'do' : ['file.01','file.02','file.03'],

In addition to letting us work with dictionaries and lists, we can also review file access and even the python CSV module if we want.

We can immediately write simple queries – "what document(s) contain the word 'prick,' but things get more interesting if you write functions to perform and and or queries - "what document(s) contain the words 'prick' or 'do'" for instance.

Why are we building this (besides as a data structure and programming exercise)? I've seen a number of references to using an inverted index when building a web search engine. In fact, I think that's something you do early on in the Udacity Mooc. I just wanted to play with information retrieval.

I remembered that there was a collection of information, including last statements from executed offenders in Texas. Someone conveniently converted it into a Google Spreadsheet. The format's a little different from our simple four file example but then there's more data. It's straightforward enough to download the spreadsheet as a CSV file and then read it with a Python program that builds it into an inverted index.

Now we have some interesting data to play with.

How many offenders used words like "sorry" or "apologize?" How about references to religion? We can do all sorts of and and or queries.

We just played with this a bit but I could see all sorts of explorations. What about taking some great work of literature and turning it into an inverted index by chapter. You could query characters or certain words and see where and when they appear in the book. A new and different way of exploring literature.

So, there you have it - an interesting little project we played with this past semester. We did it in an intro Python course but I could see it as an interesting project in AP CS A using hashmaps and lists.

Evaluate teachers like they do in the real world

This old article has been reposted a few times in my circles over the past few weeks It's from 2003 and complains about the lack of flexibility schools had back then with regard to hiring and salary. The article is old and out of date and the seniority system no longer holds for hiring but it does for lay offs. The salary scale from back then, though, is still in place.

My friend Alex asked my thought about how we might deal with evaluating teacher quality. I thought I'd share them here.

Alex listed a number of commonly tried and suggested options on teacher evaluation and why they're all flawed.

I haven't asked Alex's permission so I won't post his list and comments here. Instead let's start with the boogeyman those that attack public education always trot out – tenure.

Let's get a couple of things straight - tenure is not a job for life. For K12 teachers, it's the right to due process. That means you can't be fired without cause.

I hear the free market capitalists out there already – "in the real world, you're an at will employee, why do teachers need this due process?" - I'll get to that down below. For now, let's look at a few points on tenure:

First - you can fire a tenured teacher, you just have to document your case. This means that the supervisor has to actually do their job.

Next - people can complain about rubber rooms and the time it takes to fire one of these tenured teachers. That's because the DOE won't hire sufficient hearing officers. In a recently resolved case, a friend of mine - a terrific AP was finally returned to her post after she was railroaded on false charges. She was in the rubber room for years. The DOE kept delaying the process – all she wanted was to go to "trial" since everyone knew she'd be exonerated - the fact that she was up on charges at all speaks to why teachers need tenure.

Next - teachers don't grant tenure, the administration does. Why are all these horrible teachers being granted tenure? Stories abound of administrators granting tenure to bad teachers so long as they transfer to another school - who's to blame here?

Finally, the contract that establishes tenure is jointly negotiated by the teachers union and the city - no one ever seems to complain about the city.

Next, let's talk about bonuses and competition. People in the business world take the false assumption that everyone thinks, acts, and feels like they do. Here's a surprising fact – teachers aren't in it for the money. Sure, we'd like to make a little more - perhaps enough to actually live in a decent apartment and not need a second job to make ends meet - but no one went into teaching to get rich. Teachers are in it to uplift ALL their students and their school as a whole.

Now let's look at the model currently in vogue - value added measures. Besides the fact that they're arbitrary and don't work (see this post and this one) they're nothing like how people are evaluated in the real world.

How are people evaluated in regular jobs? As far as I know, they're evaluated by their managers and bosses - possibly also by their peers.

Funny thing - that's how teachers used to be evaluated before all this nonsense brought about by the "reformers." Of course, no one will admit to this.

Did this system work? For the most part, but not entirely. Why were there problems? Because, unlike a business, principals are not necessarily incentivized to have the best teachers.

Schools and principals are evaluated on ridiculous metrics - the same standardized tests they use to evaluate teachers. To look at a small piece – to graduate high school, kids need to pass at least one math regents and pass three years of math classes. To be "college ready" a student still only needs to pass one math regents and take at least one year of geometry or trig.

This means that as far as math is concerned, a principal is incentivized to have a math department that can get their kids through the algebra regents. Here's how this frequently plays out. The kids take algebra over two years and then passes the exam. The kids take geometry as their third year. They don't have to pass the regents, but it appears that in order to increase a student's chances of passing that exam, classes don't teach proof - the most important and hardest part of the class. Why not? Because it's easier to pass the exam by learning most of the other stuff.

Here we have a school where the incentive is to hire teachers that can teach to the algebra regents and that's about it.

That's at a school with a low performing population. In a higher performing school, the kids will do well on the standardized tests regardless of teacher quality so there's no incentive to hire the best. Even if that's what they want to do, there's no way to know if they're actually doing it.

Back to why teachers need due process. Since principals aren't held accountable in any reasonable way, teachers can't be. The end result is that teachers are subject to abuses by principals. I myself was once brought up on bogus charges raised by a principal (not a Stuy one) who wanted to make trouble. Without due process, I would be out of a job. So would many others.

The answer? To me it's figure out how to hold principals accountable. In Alex's Facebook post he mentions evaluation by college and workforce success but that has a slow feedback loop. The truth is, education does have a slow feedback loop. Schools, however, exist for along time and principals should have tenure periods of more than just a couple of years.

My feelings? Why not look at graduates succeeding in college or on the tax rolls? This wouldn't be perfect but it would be a start. While NY can't count on private universities sharing accurate information in a timely manner there should be a way of tracking if a high school graduate is enrolled in a SUNY or CUNY one, two, three, or four semesters after graduation. Create a baseline for a school and start from there.

If principals were actually accountable in a reasonable way they'd have incentive to have the best teachers. Then we can get back to the old system which was indeed the way the "real world" works.

Using Emacs - 21 - web mode

Quick post today.

The video goes over web-mode - my preferred weapon for html and all the goodies you embed in an html file.

It's multi-modal so it acts sensibly regardless of wether you're editing html, css, javascript or even templates in a single file.

Note: In the video, web-mode wasn't automatically inserting quotes. I had to set hte following variable to fix this:

(setq web-mode-enable-auto-quoting t)

Giving the final configuration I'm currently using:

  (use-package web-mode
    :ensure t
	 (add-to-list 'auto-mode-alist '("\\.html?\\'" . web-mode))
	 (setq web-mode-engines-alist
	       '(("django"    . "\\.html\\'")))
	 (setq web-mode-ac-sources-alist
	       '(("css" . (ac-source-css-property))
		 ("html" . (ac-source-words-in-buffer ac-source-abbrev))))

(setq web-mode-enable-auto-closing t)
(setq web-mode-enable-auto-quoting t)) ; this fixes the quote problem I mentioned

The web-mode documentation has more information including a nice chart of all the key bindings.

Using Emacs - 22 - emacsclient

Another quick hit today.

This video is really setting the stage for hte next two, where I talk about org-mode capturing.

Today, we're demoing the use of the emacs server and client.

Back in the old days, emacs took a LOOONG time to load. A complex conviguration could take on the order of 10 to 20 seconds. This resulted in people loading emacs at the start of the day and leaving it up forever. While load times aren't much of an issue anymore, it's still nice to leave emacs around all the time.

That's where running emacs as a server comes in.. I set up a keyboard shortcut my window manager to run:

emacsclient -a "" -c

Rather than trying to run emacs, this tries to connect to an already running emacs server. The -c creates a new frame. Instead if you use -t it will open emacs in the terminal. The *-a ""* tells emacs to run itself as a server (daemon) if it isn't already running.

Later on, running emacsclient -c will very quickly bring up a new frame to the server.

In addition to instant loading, Running emacs this way has the advantage, or some may say disadvantage of keeping buffers around forever since you just keep emacs up and running.

Using Emacs - 20 - yanking

Today's video is a bit of a follow up on the last one as well as a few miscillaneous configuration.

We got some comments on alternate ways to load a file if it exists - it's always cool to see how different people end up doing similar things.

We then talk about three little configurations. Two I found out about reading Jon Sander's (jcs) blog Irreal. Jon's blog is a terrific source of emacs info - if you don't subscribe, you should.

I go over using counsel for working with the kill-ring. The command counsel-yank-pop acts as a replamcement for yank-pop and it lets you easily yank anything from the kill ring - not just the most previously killed/whacked piece.

You just hit M-y and then either keep hitting it or use up and down to select what you want to yank.

Here's the original post from Irreal:

I then briefly talk about smoothing things over between the system clipboard and the kill ring, also from Irreal:

Finally, I talk about auto-revert-mode - this monitors files on your file system and when they change, they reload your buffer. I find this very useful to keep multiple machines in sync when I'm working with files in a could based filesystem like Dropbox or Owncloud.

Science Talent Search winners do not a great school make

Over on Facebook, my Alma Mater and long time employer, Stuyvesant High School seems to be making a big push to up it's Regeneron Science Talent Search game. For those of you who aren't familiar, that's the latest incarnation of the Westinghouse Science Talent Search from back in my day.

It seems that Stuyvesant hasn't had that many semi finalists, finalists, and winners over the past few years and isn't happy about it. The result is a bunch of social media reminding us of old winners and trying to raise funds to get us back to the glory days.

I'd rather they spent there time, energy, and money elsewhere.

When I was a young teacher, I was enamored with those types of results. When my kids did well on a USACO competition, for example, I felt pretty good about myself.

Then one day, I had an epiphany. I was looking over the results of a big USACO competition. My kids came in 4th, 5th, and I think fifteenth. I was feeling pretty good about myself but then I realized that Dan, Evan, and Jon - my three top scorers would probably have done just fine without me. Sure, I do believe that I was able to help them in their growth as both computer scientists and as people but they were already off the charts smart.

The same is true about the Science Talent Search (STS) winners. A friend of mine ran Stuyvesant's math research class for a number of years. He had a pretty impressive string of finalists and winners. How did he do it?

  • Identify Stuyvesant's top talent - the ones already most likely to succeed in the STS.
  • Pawn them off on a lab or professor.

The Stuy class, by and large, did little or nothing for these kids yet we would hold them up as shining examples of how wonderful we were.

When I had my epiphany, I started designing my intro class and focusing my efforts on the whole school population. The results? Scores of people succeeding in tech who never would have considered it otherwise.

While my super high achievers have indeed gone on to successful lives and careers, some of my most successful graduates, by any number of measures have been the students who would otherwise have gotten lost in the crowd at Stuy.

The truth is, many of Stuyvesant's STS winners would have been STS winners regardless of where they went. We may or may not have helped them along the way but STS results are not the measure of a great school.

Improving a school, any school, for all its students might not help in those bogus national ratings but that really should be what we're about.

Using Emacs - 19 - moving to a live config

Some of the videos I want to make are going to be much easier to do if I'm working in my own account where my whole file tree is available.

It's also a little annoying changing between my full emacs config and the one I'm developing here.

So, I decided to move to this config for my real config and as I make more videos move parts of my old configuration over to this new one.

The only problem is that I need some of my current config right now – things like my email setup.

This video talks about how I'm going to set things up to take care of everything.

Flask - deploying to DigitalOcean

This post points to the next three videos in the Flask series.

The first covers setting up a droplet (virtual server) on DigitalOcean. The second, deploying a flask application using Green Unicorn, and the third, using sshfs to remotely mount your files from your DO box to your local machine and how to set up DNS using FreeDNS.

To help you on your way with DigitalOcean, here are links to a few of their tutorials

In general, there are lots of great tutorials and guides at the Digital Ocean Community.

Setting up a Droplet

Deploying with Gunicorn

sshfs and FreeDNS

Using Emacs - 18 - Narrowing and iedit

I meant to cover these last time but decided not to so as to keep the videos to about 10 minutes each.

First up this time round is iedit - a really cool mode that lets you select all the instances of the marked selection at the same time and edit them all at once. By default it's bound to C-l.

For example, let's say you had the following code:

def f(somevar):
    somevar = somevar + 1
    return somevar

You could mark one instance of somevar, hit C-; and you'd be editing them all at once. When you're done, you hit C-; again and everything is un-selected.

There are other ways to modify multiple instances of a selection. You can always search and replace or use Magnar Sveen's really cool multiple cursors package and I'm sure there are others but I like iedit since it's small and simple. It also works for rectangular editing but I didn't cover that in the video. I'll probably demo it when I cover rectangular editing in general.

Iedit has limitations. For insance, since it works globally, it isn't perfect for renaming variables. If the above code snippet was part of a larger program with many other instances of somevar, iedit wouldn't, by itself, be as useful.

You could reach for a language specific refactoring package but for me, that's where narrowing comes in. You can mark a region and tell emcas to narrow-to-region which hides everything outside the region and temporarily (until you widen) treats the region as the whole document.

So, you could narrow to the section of code you want to edit and then use iedit from within.

To make things easier, I grabed narrow-or-winden-dwim from Endless Parentheses.

I think the video makes the behaviour pretty clear.


Enter your email address:

Delivered by FeedBurner

Google Analytics Alternative