YAPC::Europe: Tuesday

it's just my notes, modified for
obvious spelling errors fixes and URLs for the interesting bits. It may
contains errors. I'll post proper and scoped articles later.

Upate: Fixed broken links

Larry Wall's Keynote

scripting
languages

past, present and future

ruby
most direct competitor for perl

perl 6 mix between
pure scripting and programming language

lua,
applescript: niche player

failed: tcl (due to lack
of extensibility), *sh (clumsy addition of layers of features)

early
binding vs late binding
perl6: all method are virtual by
default

single dispatch, multiple dispatch
single:
python, perl5
multiple: perl6, dylan

eager
or lazy evaluation
haskell: very lazy evaluation
perl6:
scalar is eager by default, list is lazy

eager
typology, lazy typology:
types introduced in perl6 for the
multiple dispatch
fixes e.g: prototyping

removed
from perl6 punctuation that were not really necessary
introduce
a new one for scoping

mutable, immutable classes
java
classes are immutable -> fast
ruby classes are mutable
-> ruby slow
perl6 will have a mix

class
based OO, prototype based OO
perl6 will be classed based, but
meta data will allow prototype based OO
(see Moose in perl5)

perl6:
given ... when ... for regexp?

Selenium,

an introduction to web testing (Barbie)

benefits over
Mechanize: test javascript as it runs

Thoughworks
released
as open source on openQA

use javascript and iframes
in the browser
core runs the tests and interrogates the DOM
RC
server and core communicate via AJAX

Core, Remote
Control, IDE (firefox plugin)

Core: issues with Opera

RC:
java, requires JRE version 1.5.0 or higher
experimental
support for SSL
language hooks for
java, .Net (C#),
Perl, PHP, Python, Ruby

Mozilla same origin policy

IDE:
record/playback, edit and debug tests
include Selenium Core

cpan>
install Alien::SeleniumRC

(cant upgrade due to how versions are dealt with)

cpan>
install Test::WWW::Selenium

$ selenium-rc

use WWW::Selenium::Util
qw(serverisrunning)

Evolving

architecture - make development easy and your site faster (Leo Lapworth)

evolution

running
website
* servers improved
* architecture too
* development tools improved
* language

  • templates

hard
coded html in the beginning
now templates
Template

  • servers

from development on 1 server to 3-tier servers

svk with subversion

trac

  • tests

  • make your site faster

mod_perl (code caching)

Apache::SizeLimit (safety net)

-> set it high

-> check it

  • front/back end split

(sees Omnigraffle schema)

add caching

search result sets

individual items

lookups for info from database

lookup from external sources

put caching methods in one packages

separate cache for each backend servers?

-> share them (using memcached)

perlbal -> load balancer/proxy

mod_gzip

cache headers (expiry)

/includes/js//common.js -> can be cached forever

ensures user has version which matches html

use include file to update all pages

  • handling images: MogileFS

  • centralize

  • test

  • cache

  • kiss (esp. perlbal)

XML::Compile::SOAP (Mark Overmeer)

XML
sucks (verbose, looks simple but its not)
XML schema
WSDL,
SOAP

avoid learning XML and Schema

pure
perl, compliant, complete, validating, xml message reading and writing

use
XML::Compile::Schema;

good:
automatic
name-spaces
type structures hidden (inheritance etc)
template
generator

limitation:
only
name-space based schemas
mixed content only via hooks
schemas
themselves not validated
you need a schema to use
the module

SOAP (PayLoad - all XML -, Transport - in
application)
Payload = Body + Header ( Envelope)

two
kinds of SOAP:
Document
* well defined body
* requires longer schemas

XML-RPC
* interface quick and dirty
* SOAP::Lite special
* discouraged in SOAP 1.2

WSDL
message
structure and transports details are grouped together.

XML::Compile::WSDL

SOAP
client/server implementation still under construction

use
BigInt instead of sloppy int -> slight reduction in performances.

Gluing

a bank together (UBS) (Paul Johnson)

move lots of money around to
avoid interests or to gain interests

Cash management

CPAN

primary
development is outsourced

needs to customize the
product

needs to be integrated

database
web
servers
communications
high availability
monitoring
logging
archiving
deployment

initially
role is automated testing

perl as a development
language is not allowed in UBS
but perl to glue
thing together is Ok, then development could be done

Oracle
100s
of GB

  • Web server
    IHS: IBM re branded version of Apache

  • Communication

multiple
sources
multiple format

  • message transfer: how amount goes from what bank to what other bank in what
    currency
    IBM MQSeries

- mail

SMS
- IRC
- file transfer

pack
and unpack

use Spreadsheet::ParseExcel

use
MQSeries ; # written and maintained by Morgan Stanley people

system
handles many millions of money currency

if system
breaks, huge amount of money is lost

monitoring
-> Nagios

logs

50 GB a day
require application restart to log rotate

so
he write wrappers with named pipes, correct formatting including
timestamps

use Log::Log4Perl

Deployment:

Sun
packages

package creation

mini
CPAN burnt on CD

Extra development

internal
part base on Catalyst
(DBIx::Class, Template Toolkit)

Automated testing

Test::*

use Test::WWW::Selenium

Trexy (Nigel Hamilton)

trexy.com

remember
search trails

my trails - all trails - blaze a trial

30
millions incoming links

Sys::Statistics::Linux::MemStats

pingability.com

webmin

Template::Simple

The
Goo

perceptrons
sensors

http://blog.thegoo.org

Tech
Pub Crawl: first Tuesday of the month in London
flag-and-bell.com
FREE BEER

memcached (Leon Brocard)

network
effect
-> scaling?

temporary storage
area fro frequently accessed data can be stored for rapid access

trade
memory/disk speed

One Server:

MySQL
query cached - invalidated on write

Disk - Cache::FileCache
scales
really well
memory bound

mod_perl
only
one per child

shared memory
not as fast as
you might think

cache is separate on each

lower
hit ratio
higher miss ratio

memcached
giant
hash table distributed across machines

never
blocks
libevent
epoll/kqueue
slab
allocator
least request used
thread
per cpu (optionally)
version 1.2.x are much better

facebook:
3TB memcached

use Cache::Memcached

Pattern:
fetch
from cache
if there return

else calculate, place in cache, return

cache, not a
database
-> cant dump
-> no persistence
-> no redundancy
-> no access by id
-> ...

time to live

smart
caching
timestamps, version number in key
cache
forever

low CPU

Failover?
doest
do it for you
replace failed server with another with same ip
or
use consistent hashing

limits:
keys: max
250 chars
values: max 1MB

Testing
* disable memcached

future:

consistent
hashing
binary protocol
more statistics

http://www.danga.com/memcached/

has
to push the keys to all memcached servers

memcached, perlbal, mogileFS, Djabberd,Gearman

TheSchwartz

Net::Proxy (Philippe Bruhat)

connectors

a
connector handles the pairs of socket (one for each client)

Use?
* escape the corporate proxy
CONNECT method
(abuse of the the CONNECT, normally for SSL?)
* avoid Intrusion Detection Systems
* early stage of ssh negotiation is not encrypted and can be detected by IDS by doing a
m/ssh/

  • use hooks to hide ssh signature using one Net::Proxy before the firewall and another Net::Proxy the other side of the
    firewall to decrypt ssh signature
  • add SSL support to an application that doest support it
  • run two servers on the same port
    we want to run
    sshd and https on the same port

  • in ssh negotiation, server speaks first

  • in http/ssl: client speaks first

  • Net::Proxy uses that to make it possible

* todo:
* write a connector fully compatible with GNU httptunnel

  • enhance the httptunnel protocol to support multiple connections.

  • implement reverse connectors (as you cannot connect to
    machines
    behind firewalls at the moment)

  • implement DNS tunnel connectors

  • implement UDP connectors

  • implement a connector that can be plugged to the STDIN/STDOUT of an external process, like the ProxyCommand option of OpenSSH

  • finish the starttls connector

  • implement SOCKS connectors

Rija Ménagé

Read more posts by this author.

Subscribe to Pomme::TAB

Get the latest posts delivered right to your inbox.

or subscribe via RSS with Feedly!