Alexa Web Crawler - TamperedWithCookie Error

Posted by patjoyce
Dec 02, 2008

I’ve recently started receiving a ton of CGI::Session::CookieStore::TamperedWithCookie errors on an application I manage. All of the errors are coming from the Alexa Web Crawler.

The application is using the Cookie Session Store (Railscast) introduced in Rails 2. Upon inspection it appears that the problem is related to the newline characters rails inserts every 60 characters. Rails encodes these newlines as %0A. The alexa web crawler seems to convert these to \n. This is the only difference between the two cookies that I can see.

Rails Cookie:


_application_session=BAh7CToMY3NyZl9pZCIlZDA1ODAzY2MwZGZjNzJkN2I4NGFjZTE5OTcxNzZh%0ANjI6FnNlYXJjaF9jYXRlZ29yaWVzWxRvOg1DYXRlZ29yeQc6FkBhdHRyaWJ1%0AdGVzX2NhY2hlewA6EEBhdHRyaWJ1dGVzewciCW5hbWUiFUF1dG8gLyBUcmFu%0Ac3BvcnQiB2lkIgYxbzsHBzsIewA7CXsHIgluYW1lIhNCZWF1dHkgLyBJbWFn%0AZSIHaWQiBjVvOwcHOwh7ADsJewciCW5hbWUiEkJvYXQgLyBNYXJpbmUiB2lk%0AIggxMTVvOwcHOwh7ADsJewciCW5hbWUiFUJ1c2luZXNzIC8gQWRtaW4iB2lk%0AIgY5bzsHBzsIewA7CXsHIgluYW1lIhRDb21wdXRlciAvIFRlY2giB2lkIgcx%0AMW87Bwc7CHsAOwl7ByIJbmFtZSIVQ3JlYXRpdmUgLyBNZWRpYSIHaWQiBzEy%0AbzsHBzs;

Alexa Crawler Request Cookie:


_application_session=BAh7CToMY3NyZl9pZCIlOGJlNGQ2ZGYzYWZjODRhZGI4YmNlMWUxZTkwNmNl\nYjA6FnNlYXJjaF9jYXRlZ29yaWVzWxRvOg1DYXRlZ29yeQc6FkBhdHRyaWJ1\ndGVzX2NhY2hlewA6EEBhdHRyaWJ1dGVzewciCW5hbWUiFUF1dG8gLyBUcmFu\nc3BvcnQiB2lkIgYxbzsHBzsIewA7CXsHIgluYW1lIhNCZWF1dHkgLyBJbWFn\nZSIHaWQiBjVvOwcHOwh7ADsJewciCW5hbWUiEkJvYXQgLyBNYXJpbmUiB2lk\nIggxMTVvOwcHOwh7ADsJewciCW5hbWUiFUJ1c2luZXNzIC8gQWRtaW4iB2lk\nIgY5bzsHBzsIewA7CXsHIgluYW1lIhRDb21wdXRlciAvIFRlY2giB2lkIgcx\nMW87Bwc7CHsAOwl7ByIJbmFtZSIVQ3JlYXRpdmUgLyBNZWRpYSIHaWQiBzEy\nbzsHBzs

I’m not the only person having this problem. Any ideas?


Error - rake ultrasphinx:configure

Posted by patjoyce
Nov 07, 2008

A new Web Designer I work with was running into an error when attempting to configure ultrasphinx for his local Rails development environment.


rake ultrasphinx:configure

rake aborted!
Anonymous modules have no name to be referenced by
/var/lib/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:402:in `to_constant_name'
/var/lib/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:214:in `qualified_name_for'
/var/lib/gems/1.8/gems/activesupport-2.0.2/lib/active_support/dependencies.rb:477:in `const_missing'
/home/deploy/apps/stylespotted/releases/20080131024018/vendor/plugins/ultrasphinx/lib/ultrasphinx/configure.rb:33:in `load_constants'

I was baffled for this for a few days. We finally figured it out this afternoon.

He is a UI developer, and new to Rails, so hadn’t created his DB. The fix was simple:


rake db:create
rake db:migrate
rake ultrasphinx:configure

And he had a working development.conf for Ultrasphinx


Freezing to Rails 2.0.4

Posted by patjoyce
Sep 21, 2008

One of our applications is still running Rails 2.0. I’m planning to upgrade to 2.1 soon, but in the meantime I wanted to upgrade to 2.0.4 to get the patch for the REXML DoS vulnerability. We freeze our version of Rails (and so should you) but I couldn’t freeze in the normal way:

rake rails:freeze:edge

This resulted in a message that the release was not found so it checked out the head instead.

I managed to freeze by running the following two commands:

sudo gem install rails --version 2.0.4
rake environment VERSION=2.0.4 rails:freeze:gems

alias versus alias_method

Posted by patjoyce
Jan 22, 2008

Last week Ian and I were running into a problem with alias_method. It turned out that we were passing the method name to alias method as you would to alias, instead of passing a string or a symbol of the method name. Here is the correct usage of alias and alias method

  
  alias new_method_name old_method_name
  alias_method :new_method_name, :old_method_name
  alias_method 'new_method_name', 'old_method_name'
  

Here is the blog post that pointed out our error. It is in Spanish, so for the first time ever all that time I spent learning Spanish helped me as a programmer ;).

For the benefit of English speakers I decided to translate the blog post. Here is the translated text:

alias and alias_method do the same thing: copy a method and assign it a different name.

  
  class Test
    def test
      puts "hello" 
    end
    alias test_copy test
    alias_method :test_copy2, :test
  end

  Test.new.test        # > "hello" 
  Test.new.test_copy   # > "hello" 
  Test.new.test_copy2  # > "hello" 
  

That piqued my curiosity to know the difference between alias and alias_method. It turns out that they are the same, except that:

  1. alias is a reserved keyword in Ruby
  2. alias takes the method identifiers as parameters, without the need to use symbols or strings (for example in “def method_name”, method_name is an identifier and not a string or a symbol) This behavior can be pretty confusing when you’re starting out.
  3. alias_method is a method of the class Module
  4. alias_method takes its parameters separated by comma, just like any other method.

The consequences are simple:

  • alias_method can be redefined, and alias can’t.
  • alias can be used incorrectly (outside the context of method definitions)

By being a method of Module, alias_method makes it easier to use it correctly: in the context of method definition of a class or module.

Conclusion: In the majority of cases, alias_method is what you need.


I'm a Rails Contributor!

Posted by patjoyce
Nov 21, 2007

My first patch was committed to the Rails trunk yesterday. I’m pretty excited. The community on the rails-contrib IRC channel (particularly Rick Olson) were very helpful. Hopefully I will be able to contribute more in the future.

Patch 8174 fixes a bug where calling a dynamic finder on a has_many association ignored the :order of the association.

Background:

ActiveRecord Collections (has_many, has_many :through, and has_and_belongs_to_many) allow the use of find and dynamic finders just on the members of that collection.

Problem:

The find method behaves as expected, returning objects in the order specified in the :order parameter of the has_many association.

However, dynamic finders (find_by_xxx and find_all_by_xxx) do not respect the :order parameter passed to the has_many association.

Example:

A post has_many :comments, :order => ‘id desc’

my_post.comments.find(:first, :conditions => “comments.author_id = 1”) will return the comment with author_id 1 with the highest id.

However, my_post.comments.find_by_author_id(1) will return the comment with author_id 1 with the lowest id.


Wow.

Posted by patjoyce
Sep 28, 2007

Me and Ian attended RubyEast today. It was a good conference, obviously much smaller than RailsConf, but a nice break from working on the startup and a good chance to meet other rubyists. I’ll write up a longer recap later in the weekend but for now I just want to post one quote I found interesting.

Before his presentation Obie Fernandez was asked about the financial incentives for writing a book. He’s a good person to ask as The Rails Way is coming out soon and is pretty much preordained to be a hit by technical publishing standards. His answer:

“I will probably make a quarter of a million dollars a year for the next several years because I wrote the book. Not from the book, but because of it”

What a remarkably candid response.


Confusing Rails Error Message

Posted by patjoyce
Sep 18, 2007

ArgumentError: Admin is not missing constant UsersController!
    method load_missing_constant in dependencies.rb at line 240
    method const_missing in dependencies.rb at line 452
    method send in dependencies.rb at line 470
    method const_missing in dependencies.rb at line 470
    method setup_without_fixtures in users_controller_test.rb at line 11
    method setup in fixtures.rb at line 576

Note to self: the above error message normally means that you are missing the scope prefix and the :: operator. So in the above example the error would be fixed by prepending “Admin::” to the reference to the class “UsersController” on line 11 of users_controller_test.rb.


Rails Environments Gotcha

Posted by patjoyce
Jul 23, 2007

I’ve been working on an application that sends emails. (You’ll hear more about the application very soon) The emails often contains links to the application. In development and test I want these links to point to localhost:3000, and in production I want them to point to mydomainname.com. Therefore, the ActionMailer class needs to know the root of the application.

So, in environment.rb I added the following line:

APPLICATION_ROOT = "http://localhost:3000"

And in production.rb I added the following line:

APPLICATION_ROOT = "http://mydomainname.com"

Unfortunately, when I deployed to production the emails were being sent with http://localhost:3000 as the base for links.

I checked my mongrel config and everything seemed to be set up right for production. I checked the logs and saw that the application was running as production.

Then I ran script/console production, saw the following error message, and immediately got an idea what I did wrong.

production.rb:19: warning: already initialized constant APPLICATION_ROOT

APPLICATION_ROOT was still returning ‘http://localhost:3000’

The problem was I was counting on settings in production.rb to override those in environment.rb. And they do except I defined my setting as a CONSTANT.

I moved the constant declaration out of the environment.rb and into both development.rb and test.rb and everything now works fine.

The thing I still don’t understand is that according to page 330 of the PickAxe Ruby is supposed to allow you to alter the value of a constant.

Anyone know why the constant’s value didn’t change in my config files?


July NOVA RUG Recap

Posted by patjoyce
Jul 18, 2007

I went to the NOVA RUG meeting tonight. It was my first NOVA RUG meeting and I’m glad that I went.

There were three presentations, one on Ruby Mocking Libraries, one on HAML and SASS, and one on working with legacy databases in Rails. My thoughts follow.

Mockfight

Patrick Reagan, Director of Application Development, Viget Labs slides

I’ve heard a lot about mocking, but as someone who is still pretty new to test driven design I haven’t used it before. This presentation really showed me the benefit by walking through the example of testing a method that tests if a given URL exists. All examples were given in both FlexMock and Mocha.

Patrick has used both FlexMock and Mocha. He is currently using Mocha. He said that they are roughly feature equivalent so it comes down to a matter of taste regarding syntax style. From the examples I thought that the Mocha syntax seemed more natural.

HAML and SASS

Devin Mullins

HAML (HAML Aint a Markup Language) is an alternative to using erb for view templates. HAML is less verbose than rhtml and uses indentation to denote nesting and end tags. It generates well indented, valid xhtml. It is an interesting idea, but I haven’t ever felt real pain with writing xhtml and erb, so I doubt I will use it.

The fact that indentation is significant is reminiscent of Python. A lot of people were complaining about that similarity because they don’t like it in Python. I have wrote a little bit of Python a few years ago, but not enough to have a real opinion. That said, I think that you should always have consistently indented code or markup so I don’t see how a language that enforces that is a problem.

SASS (EDIT: SASS Aint Style Sheets Syntactically Awesome StyleSheets) is an alternative to css. It is really cool. It has a simple syntax and allows you to define constants, include other files, and eliminate duplication by nesting rules. I like CSS, but the inability to declare constants for things like border colors has always driven me crazy. SASS looks like it was designed specifically to adress all my pain points. I think that designers would love this.

The talk centered on examples, namely on Devin converting the scaffold generated view from rhtml to HAML and on showing the use of SASS.

I don’t see myself using HAML. I’m definitely going to investigate SASS.

Working with Legacy Databases

Matt Scilipoti, Possiamo Consulting, LLC

The talk centered on his experience in working with legacy databases. He is working with a large SQL Server DB that is used for protein sequencing data.

I am tired of typing and I don’t plan on having to work with legacy databases from Rails anytime soon so this is going to be short.

He started out specifying all the non standard schema stuff in his active record models. That grew painful as he had to specify the keys and table names with each relationship.

Next he moved to using Views to create an interface to the legacy DB that conformed to rails naming conventions. This was an improvement, but still sort of a pain.

Now he uses a class method whose name escapes me at the moment to handle the conversion. If anyone has slides or knows what it was that he did please leave a comment and I’ll update this section.

He also has started a Ruby Forge project Working with Legacy Databases in Rails. He will be releasing his class method as a plugin shortly.

Check it out at http://rubyforge.org/projects/weldir/

It was a good, and at times funny, talk. I learned quite a bit and am glad I heard it.


Impromptu Open Mic (Rails Security: Theory and Practice Cancelled)

Posted by patjoyce
May 20, 2007

Dan Moniz

The speaker is a no show. Aint that about a bitch.

Impromptu Open Mic (I love that you can have a talk cancel and have 20 people immediately ready to talk about something cool that they threw together over the last week. I just can’t get over the energy and ingenuity of this community):

Zed Shaw. Utu

Author of Mongrel. Talking about his new chat project utu which is a replacement for IRC. That is based on hate.

Adding Strong Identity, Reputation, and Retribution,.

The internet right now is like Detroit. The cops are worse than most of the criminals. Going for a Giulani style cleanup of the internet, but he’s not like Giulani.

Is there going to be redemption? If he can set it up so you can work off hate via charity. No love option to prevent bots.

Zed is a pretty amazin speaker.

Treetop

Nathan Sobo

Parser generator – basically parser based on regexes. Runs in linear time. Doesn’t require lexing.

Cool. Pretty clear to read. Easiest parser grammar to read i’ve ever seen.

GPL’d right now.

gembuilder

creates binary versions of gems. Will be on rubyforge soon.

ActiveWarehouse

Research on how to use DWARF algorithm to return answers really quickly. Tree data structure to handle multiple dimensions for tens of millions of rows. Implemented in ActiveWarehouse. What they found out is that Ruby is slow for computation and ActiveRecord::Base is slow, but ruby is so expressive it lets you right the algorithm easily.

embedded_actions plugin

Partials require that you load the data in eac controller that uses it. this plugin replaces partials with a full controller action/view that can be embedded into a page. Makes it easy to cache and invalidate caching.

Available soon at http://nycrb.rubyforge.com

Fuzed

Mongrel is awesome, but sort of a pain to deploy. Writing a system in ERLang that handles that whole stack.. As fast as Mongrel, but dynamically configurable. Been working on it for a week. They’re going to use Amnesia (distributed fault tolerant DB that comes with ERLang) that they’re going to use code.

Integration

This guy heard “we need to talk to ASP.Net and Java” He demos some code really easily talks to them over SOAP and it all worked fine. Rails uses multiple DBs connects to Salesforce.com via WS-*

assert_request plugin

Lets you assert the type of request in controller actions. If you get a request that doesn’t match. They use their functional tests to make sure that actions that they don’t want get bounced. Doesn’t require AR models.


Mapping Rails to Legacy Systems

Posted by patjoyce
May 18, 2007

Devon Jones, Architect of Web Systems, Vonage Stephen Becker, Developer, Vonage

They’re going to talk about the way that they have integrated with their legacy systems.

Rails at Vonage
  • Subscribe System
  • VFax
  • A bunch of internal things like Metrics Mole to show business people decision so that they can
  • RESTful services

Last year facing a software crisis. Rapid customer growth led to software developed quickly and crufty.

Large projects are a problem. They try to do small programs and projects, deliver on the order of months not weeks, and have 1 or 2 features per app, with transparent interfaces so they can chain them together.

When you have a system that is hard to change eventually you decide to replace it. Traditionally you go with a long project trying to replace the old one and owing to scope creep you end up trying to do to many things. (Sound familiar to anyone? PAS?)

Working on a legacy system is like working on a car while it’s going 60.

He is citing the Standish Group study. This has got to be the most cited study ever.

Change gradually and consistenly because people fear change, the smaller a project is the less risk there is, allows for rapid course adjustments, and it shows success early and often which helps sell stakeholders.

He just quoted ESR from tAoUP which is funny because I was thinking how much what they are describing sounds like the unix way.

Forget code reuse look for program reuse. Simple, transparent interfaces like REST (or unix pipes) make it easy to debug, test, and chain programs together.

Ripping apart a lafre legacy EJB system.
  1. Find what specific feature you need\
  2. Wrap it in a RESTful interface
  3. create a test suite for that RESTful interface.
  4. Implement the RESTful service outside of the legacy system

Suggestion 1:

Use the system. See what features / services are used. Step through in a debugger until you find where the real logic is.

As soon as you find it drop a RESTful service around it and then you can build the rails front on that.

Now you can make the legacy code use the RESTful service.

Now you have enought understanding to build a Ruby functional test suite for the RESTful service. U

Use the functional test suite and JDBC logging to figure out what is done in the old DB.

Now you can tun off the old code. The rest of the legacy app is still there and is still crap, but represents future targets.

Active Record in Legacy DBs

  • Database Degradation
  • Documentation
  • Nonstandard DB schemas
    • 8 different ways to represent false in the same column
    • terminated spelled 6 different ways in the same column
    • Dates stored as varchars
    • csv data in columns

You can define custom module methods to fix this. Exclude columns

User Interface Integration
  • Transparent Proxy: Noodle, Transproxy and Squid – Less legacy change
  • Single Sign On Keys – Alternately you can use SSO keys. – More legacy change

It worked at Vonage. They get more maintainable code out quicker, it’s well tested and has much fewer lines of code.

They repurpose old data. They have project tracking tools, test harnesses for java code written in rails,

Where does Rails fit?
  • Rapid Prototyping. You need something small and quick this is the framework.
  • Mission Typical tasks – Applications that are a little more forgiving, not quite “mission critical” things
  • Cost analysis – Development time vs. hardware. Hardware is cheap, developers are expensive.

What are they working on: composite keys

Resources

Questions

What do they do with their DB?

What they try to do is create clean services, then when all DB clients are using the service they can clean up the data without fear of breaking existing apps.

In terms of risk management, does the slow migration force your hand in terms of design?

A little, you have to maintain compatibility with legacy services.

Are they working on a API. eg. can we get access to our voicemail

They’re working on it, but management isn’t sold quite yet.

How do they do SSO?

Most solutions aren’t very cross language / platform. They rolled their own. Simple, textual interface.

From a deployment operations perspective how do you deal with the many small apps?

Take a look at JRuby. You can use your existing deployment strategy by using WAR files. If you don’t want to push to J2EE. Use Capistrano.

My take: Automate everything and I imagine that you’ll be fine.

When a Rails app shares with a legacy how do you use fixtures?

His opinion is that you shouldn’t use fixtures for testing. He is in the mock camp because he doesn’t think that a fixture based test isn’t a unit test. If you want to use them have the DBA create a separate schema with the same structure but no data. Developer: suggests kissing DBA ass.

A lot of the issues with legacy work are building relationships and bringing people who control the legacy systems don’t want them to change.

Do you gather business rules or reverse engineer?

The code is the business rule. People get confused about rules, the code doesn’t.

How do you maintain business owner interest in infrastructure improvement?

Deliver early and often shows real benefit. Once you can show them that they can get features they want quick they won’t go back. Projects like this thrive on agility. They use two week development cycles. Ability to readjust priorities is a huge win.

Is there an online communithy around these issues

Enterprise Rails Yahoo group??


Full-stack webapp testing with Selenium and Rails

Posted by patjoyce
May 18, 2007

Alex Chafee and Brian Takita, Pivotal Labs

slides

They’ve done 15 ruby on rails app in the past 18 months.

Selenium scripts the browser (Mozilla, Safarie, IE) so it allows you to test the whole application. The use Selenium RC that lets you automatically start / stop / control any supported browser that allows you to do cross browser testing in one test run which is useful for CI.

What you want is a whole lot of unit tests (Test::Unit / jsunit) and a few integration tests (Selenium) that test the whole thing.

Demo of them running their selenium tests that are integrated into their rake build. Selenium lets you test AJAX / DHTML stuff.

Selenium IDE is a firefox plugin that lets you record selenium scripts by clicking through your web app.

You can write selenium tests in an internal Ruby DSL! You can also use Java, .Net, Python, Perl. Because you’re writing in ruby you can access activerecord objects, etc as well. You can stub email by using action mailer.

Don’t forget to extract common actions into a DSL (eg. login, logout) You can provide QA with login_as_user, login_as_admin etc and then they can either write low level (click on this button) or high level actions. This makes tests a whole hell of a lot more legible, debuggable, and generally easier to work with.

Showing the architecture of the Selenium RC Architecture. It is a bit complicated, but I imagine that it needs to be to pull off everything that they are doing.

Question: Can you only use Webrick?

No, they’ve used Mongrel before, but it doesn’t have a huge perf impact. Yeah, it’s not the exact same production stack, but that is by design. They want to be able to load fixtures and stub email, etc.

They have a virtual farm of parallels instances running on a mac mini that lets them do Unix, Windows, Mac Safari, Firefox, IE. Idea is to let them have there

Because AJAX and DHTML are asynchronous and dynamic they are hard to test. The way you do it is using wait_for (which runs in ruby) and wait_for_condtion.

Good slide showing the types of different tests. You only want a few Selenium tests. Why? they are slow, and they can kill CI. You do want to use it because it is incredibly powerful. They’re experience is that integration tests are the most important. It is a good idea to have a smoke test, just makes sure it doesn’t blow up. Even a single selenium test that walks through the whole app doing just the nominal case can have a huge test. Don’t use for every edge case.

To counteract the speed issues, break into two suites a slow suite and a fast suite. So run the fast suite before checkin, but kick the slow suite in the CI. so if something in the slow suite breaks you’ll know in the hour.

Selenium tests are less DRY than unit tests just as unit tests are less DRY than prod code. Selenium tests are supposed to exactly mimic user interaction.

Questions

I appreciate the value of this in a large team. Do you recommend this for a small team?

Still recommend Selenium RC so that you can do cross platform testing. Also it lets you write your tests in Ruby. Can still be add a lot of value for a small team.

Pros

  • Top-down
  • Test passes features done

Cons

  • Brittle
  • Slow

How to get started?

OpenQA. Documentation will be improved as they open source their rake tasks.

Can you script x/y coordinate clicks to test Flash?

Yes. Testing Flash with Selenium is apparently a huge pain.


DTrace on Rails BoF

Posted by patjoyce
May 18, 2007

Just got back from the a birds of a feather session about “DTrace” and the work that Joyent has done to instrument Ruby and Rails. I’d read about DTrace, but listening to the guy who used it to knock 80% off of Twitter’s CPU usage is pretty cool. Afterwards I got a chance to talk (and by talk I mean stand next to and listen because I don’t have anything to add) one of the guys from Joyent, some of the developers from Kongregate talk about XMPP. The guy from Joyent talked about watching the TCP window shrink via DTrace while a Ruby XMPP thread was blocked. I’m botching the retelling of the story now, but it was incredibly impressive that you could watch the internals of the TCP protocol at work.

I just can’t describe how unbelievably cool DTrace is. You can time all your rails function calls, then all the system calls they kick out to the db, then all the internal calls in the db and all the way back up. With this level of instrumentation you can pinpoint exactly where your bottleneck is.

DTrace is going to be in Leapord along with a really sweet GUI called Xray. I just hope I have an app someday where I’ll have enough load that I need it.


Harnessing Capistrano: Notes Part 2

Posted by patjoyce
May 17, 2007

Jamis just made a joke about the sea of macs. I guess I’m not the only one who thought that it was noteworthy.

there are before and after hooks as well as a bunch of callbacks. start and finish are only tied to top level tasks.

Upgrading to Capistrano 2.0

shouldn’t be too bad unless you did a lot of customization.

  1. capify . to set up.
  2. 3rd party extensions like mongrel cluster won’t work.
  3. overridden or extended tasks might blow up
  4. some of the variables have been changed
    • there is a “compat” mode that provides a facade of tasks with 1.4 names delegating to 2.0 cap -Ff compat
    • seems like compat mode would be a good way to figure out what has been deprecated in your resumes and what you need to change.
  5. cap -Ff upgrade upgrade:revisions dropped the revisions log in 2.0. There will be a file in each release directory that says called “revision” that specifies the revision that was built.

Done with the slides.

Questions

Why wasn’t Capistrano developed as an extension to rake?

Similarities are mostly superficial. Rake is dependency oriented. Capistrano is declarative. Rake for local, Capistrano for remote and hasn’t encountered much overlap.

What is up with Ezra’s shell?

I don’t really know what they are talking about but I’ll need to know.

What is the recommended way for setting environment variables?

There is some method that you can pass

What other types of cluster admin tasks do you do?

Cap isn’t moving away from deployment. Deployment will always be a central task, but Cap 2 makes it much more modular and more accessible for sysadmins. 37s uses for troubleshooting. Their sysadmin keeps cap shell all the time. Uses to check filespace etc, which was a big deal before they moved to using S3 (they were running out of space) Someone uses for their config files. They set their apache config files as erb templates.

Design question: Dependency checking is a very hardwired syntax have you thought about a more general way to do dependency checking?

He didn’t know there were other options. Just wanted to get something working. Open to someone coming up with a better way to do it. Need to support windows as a platform to deploy from limits choices. Example Jamis had to declined patch do to cd syntax differs on windows from POSIX.

Most deploy example seem to be simple. Where can you go for examples of more advanced stuff?

Eventually: Capify.org The wiki is a cool tool also. Right now the best place is the list. Another good place is blogs example. Tim Lucas. Capistrano is soft of hard to learn. There is an O’Reilly book that covers the basics, but is small and doesn’t get into detail.

How do you parametrize tasks?

Suggestion: use either environment variables or capistrano variable. Do it the same way you would parametrize a Rake task.

Isn’t it kind of messy to use environment variables?

The problem isn’t so technical, but how do you specify those on the command line. Jamis likes the env variable. Dirty on windows (what isn’t? that’s mine) A task that needs to take many parameters is a smell. Can prompt by using lambda.

Demo of “CocoaCapistrano” little ssl like thing

How did you record your scripting demos?

wrote a script called text_movie. The demos were smoke and mirrors.

What do you do with long running migrations (order of days)?

He doesn’t know. There longest migration was 3 hours. The guy asking now just runs them and prays. Another guy uses Capistrano to kick off the processes and has a separate system to do the migration and monitor it.

Deployment Migration. Can’t running deploy without migration break?

Yes. But safer than messing up data. Very hard to stop alter table on a row with 4 million rows. You can roll back if you blow up your app by not doing a migration. If you think deploy should migrate by default you can make it by putting an after hook. This is what they do at Shopify.


Harnessing Capistrano: Notes Part 1

Posted by patjoyce
May 17, 2007

First off, I can not overstate how overwhelmingly popular the mac is. There are probably 100 laptops in the room right now and I see 4 non macs. It was the same this morning in the JRuby session which is where you would expect to find the more corporate people. Those who do have non macs appear to be running Ubuntu. I think I’m the only person with a windows box in the room. And I am a Mac user who only has this dell because work gave it to me. Not that this is news or anything, just thought I would mention it while I’m waiting for the room to fill out.

Presentation Slides

Capistrano started out as a deploy tool when basecamp got pushed to multiple servers. Now it is used for more, It can be used for
  • ad-hoc monitoring
  • server maintenance (they edited nfs mounts against 6 app servers with one command)
  • troubleshooting – grep logs across multiple servers

Capistrano is a tool for executing tasks on remote servers.

Capfile

  • internalruby dsl similar syntax to rake, but not compatible.
  • specifies what servers to run against and what task to do.
  • Gateways – Allow you to manage a cluster by tunneling through your 1 gateway server. Useful when the individual cluster servers aren’t publicly accessible.
  • Can specify multiple roles (a role is a collection of servers) and limit takes to particular roles.

invoke lets you run a particular command against whatever servers you want (even without a capfile) shell lets you run a shell (better than invoke because less typing and faster bc connections are cached) this is experimental so beware, but I can see how it would be crazy useful.

Namespaces

  • new in Cap 2.0.
  • default tasks, task named :default can be executed as the namespace name.

Variables

  • set, eg. set :deploy_to, "$HOME/rails-apps/#{application}"
  • -s foo = bar (sets after recipes loaded)
  • -S foo=bar (sets _before recipes loaded)
  • in capfiles check if the variable exists before setting to avoid conflict with command line parameters

Transactions

  • not overwhelmingly robust, but is pretty cool and definitely helps

Deployment

  • love how opinionated about the need for source control they are. I agree 100% if your code needs deployment it needs source control.
  • looks really easy. Like everything with rails I’ve dealt with well thought out.
  • deployment tree looks like a good way to manage releases

Dependencies

dependency checking is a good idea. lets you check things about servers like whether the expected directories exist, their permissions are correct, and subversion exists. you can add your own dependencies as well. Custom dependencies let you check if arbitrary directories, gems, permissions, PATH. this is effin brilliant. still sort of experimental, only does what Jamis needed but will be growing.

script/spin tells how to start server. see slides for details. compare to process supervision tools like init, svscan, runit. You can override deploy:start

some tasks use sudo by default.

deploy:cold – first deploy. laughs at “it never works the first time.” Problems are normally that the webserver / mongrel / fcgi setup.

cap deploy will push a new version. cap deploy:rollback does exactly that. Note: deploy doesn’t run migrations by default. Jamis thinks any prod DB changes should be explicit. I agree (not that my opinion is very valuable ;) )

Deployment Strategies

  • different ways of deploying apps. pretty cool. extensible so you can write custom ways of deploying apps.

Can use other types of source code control other than subversion. none of the stuff we use at work is supported. lots of other things are.