Milagro Collaboration Meeting

1998 October 18-19, Los Alamos

Gus –

Reports that we are about a month behind. Most of that was the cable testing. All tubes, but 8 are in.

Don –

Cover – it worked. Pictures are shown… "Simple to use – Easy to Abuse"


    1. Rainstorms
    2. Up/Down in the winds
    3. Emabattled Baffles / pummeled PMTs
    4. Misues of controls & tensioners
    5. Winterization
Replace end rods

Seal Winches

Lube & antilube

Ropes & webs left out?

Lightning Protection system – does it work? Two strikes within 100 m, but no monitor.

Last May there were 4 days with strong positive strokes due to the Mexica fires

Winterization –

Lower tension so that ice won’t build up

Dorfan has given up on his heating scheme

Get a field mill

Strike Monitor should be made

Install ACTs & outriggers in a consistent manor

Sell system to Cy

Allen –

Test of String Theory –

October 1995 Survey on Pond bottom

July 1996 Some overlap qit 10/95 + some point some slope

Grid contracted – this is from survey and observation - The shift is about 5cm.

0.17"/grid spacing

The contraction was removed and the surveys (1998 v 1995) compared.

To the level of .02 degrees they agree in rotation.

As to height – 8 to 10cm over the pond bottom. This is a rotation in the Y Z plane.

This is 0.5mr +- .013mr

A measurement was made while Grito was being emptied to measure the effective height of the water. WE ARE BEING CONSISTENT with our definition of UP!

The 4 marks give a 0.98mr shift. You had to rotate the survey(1998) 0.9mr and the 1995 Grito by 1.4mr.

This was used for the string length calculation and it makes the data consistent without a systematic shift.

USING THE MUON layer they took actual measurements and found that the rotation was wrong by almost exactly the rotation put in. They then went and took it out by adjusting links in the air shower layer. The muon layer retains the 1.5mr shift.

Diane – PMT testing –

Failure Criteria

Resistance measurement 20 – 25 Mohms

Usually wet connectors

Charging – tripping HV supply

Pulse shape

Overall failure chargindrate –

18- sep 2 bad muon only

25- sep 4 bad all tubes

5- oct 13 bad tubes

15- oct 13 bad tubes

David – Electronics Status

Analog and digital boards are ready

All channels tested

Cables from pub connected

Cables to TDCs are ready and waiting

Multiplicity trigger is ready & tested

TDCs are ready

HV fully connected

Two spare pods on order

Scaler system is pretty well ready

Software work still needs to be done

Spare clock module being built

Gus We need 300,000 gallon

We have to evaporate 80k gallons before we can start to use

We have ordered new nozzles to improve evaporation from 6k gpd to 14kgpd

Pond reentry on 10/27

Michael Water system is working – not leaking. CY The Fence – 2500’ – 1000’ of 8’ chain link on the north side

1500’ of bard wire

Scott’s fence $25k

Ameican Fence $12.7k – went with them its done!

Cable plant

In pond – Muon layer we reused the Grito cables after extensive testing.

Shower layer we have new cables

In Pub

Patch panels connecting pond to pub – standard 100’ RG59

Rich- Calibration – Same hardware – new fibers (single cables Kevlar wrapped)

None of the fibers yet tested in situ

Wyston – Baffles – the went in well, but many are squashed by the cover. Gus – Remaining Tasks


Peter – at present water level

Installed 18 fiber "yo-yo’s" – this keeps them in place while the water is filling 6 were tangled 2 were fixed from boat

Fix 4 laser ball tangles

We should carefully inspect all shower layer tubes

Test all laser balls - There is a plan to test them by the Fleishers.

Muon PMT replacement? Requires scuba. (Spring)


Begin physics runs. (3/1)


Construction of Milagro Facilities
Dr.Jordan Goodman
October 18, 1998
FAS Number: 01524988 
Nsf Grant: PHY9119254
Indirect Cost:  0%
Funds Received:
Remaining on
New Requests
Materials/ Supplies
Santa Cruz
computer+ electronics
New York
Santa Barbara
This includes the tape library, cpus for the Challenge and the Fence 





Use exiting tanks

No plumbing

Gravel Fill and Grade sites

Use existing PMT casing with a new mounting

Deploy under LPS for now.

Prototype Design

Hang Tyvek on internal PVC frame with panel rivets

Gravel bases for foundation with buried culvert for muon paddles

Aluminum palte for manhole

Construct PMT mounting assembly from simple parts

Run cables underground

Need level ground

Digging seems to be hard

Fill seems to be best option

Hard to get bobcat to locations

Tank Construction

Cut manhole cover – clean it out

Construct PVC frame within outrigger and fasten side and bottom tyvek – panel rivets

NO GLUE will work

Place outrigger on foundation

Mount PMT assemby..
Fill with water.

PMT is mounted on a springy mount so that freezing wont break it.

Changes needed for 200

Custom designed tank

Todd –

Proposal –

1st year complete 200 outrigger array

maybe a floating ceiling

2nd year Upgrade computing system

Working Group Discussion - Jordan Goodman

What do we need? A tentative list of working groups (after iteration):
Operations (Water, Pond, EMS, Maintenance, Shifts)

Code and Data (Reconstruction, DAQ, Online Analysis, Data Management)



Outriggers and Other Upgrades

Solar Physics DAQ

These groups should meet (at least by phone) every other week or so, with minutes taken and reports made to the whole group. Disagreements should be ironed out in the small groups. Working groups have been proposed before, but they've never really worked. Why hasn't it happened up to now? Will it happen in the future? The people in charge of the groups should make sure the groups meet and the process works.

Everybody should be a member of one or perhaps two groups, and everyone should be able to do physics analysis on the data. We'll sign people up to these groups tomorrow.

Solar Proposal - Jim Ryan

Since last year - we got new hardware for the DAQ, which helps us. But, we also have a new instrument design (no igloos), which gives us no hardware muon identification. Instead we have to do software muon identification, and some progress has been made here. We also have completed new simulations on the muon effective area, which strengthen the case for Milagro as a solar energetic particle detector. And, most helpful, are the positive Milagrito results on the ground-level event on November 6, 1997. We are ready to submit the solar proposal, subject to internal review after some wordsmithing. We're asking for about $1M for construction, operations, and analysis. Submit in another month.

Engineering Run - Jordan Goodman

We don't fully "turn on" Milagro until we've understood much of the electronics, daq, calibration, reconstruction, etc. We should take data for short periods of time while experimenting with various aspects of the system until we're confident that we can turn it on and understand it. But how long will this take, one month, three months? What do we miss in the meantime? What kind of data do we write? Raw, or processed? Do we wait to "turn on" until we're confident we can throw away raw data? Difference between engineering run and real data taking is that during the engineering run you can stop the run in order to fix things. During the engineering run you focus on doing things that will serve the experiment's long-term goals. We need to have a list of those things that need to get done during the engineering run. Here's Gus's list - - -
DAQ - complete testing  11/1 
Data formats  11/14 
Write data to disk (non-HDF)  11/14 
Calibrate - slewing, tpeds, pes, etc  11/21 
Begin engineering runs  12/1 
Database complete 
Run control 
Online analysis 
Begin physics runs  3/1 
You can't do an engineering run without taking data, of course, but we have to be prepared to shut things down as we try things to improve the data taking.

Results from Milagrito

Markarian 501 and Crab, etc - Andy Smith

9 billion events - 9 Terabytes, 478 DLT tapes. Reconstruction done by Morgan, Kelin, Stefan, and Andy. All was done with the v23 reconstruction with 4ns/100m curvature correction. Processed data were inspected for errors and completeness, and concatenated into single file for each run. It took 2.5 months to reconstruct all Milagrito data into 144 GBytes, 8 DLT tapes. Stripped off Moon, Mrk 501, and Crab in dec bands, +/- 5 deg dec bands, zenith angle less than 60.

Did time sloshing background analysis, with 10 background events for every signal event.

Moon: see ~7 sigma deficit in vicinity of moon, offset and smeared due to Earth's magnetic field. Morgan will discuss details later.

Markarian 501, see marginally significant excess at 3.2 sigma, and on Crab we don't see anything (-0.2 sigma). In the Crab field there is also a small region to the north of the Crab with a significance of 3.2 sigma. So what we conclude is that we might have seen Markarian 501 - but this would not qualify as a discovery.

Monte Carlo results/sensitivity, trigger rate - Stefan Westerhoff

Effective area for both Milagrito and Milagro, and predicted sensitivities on Crab and Mrk 501, as well as predicted event rate for Milagrito.

Geometric area is equal to effective area at ~2 TeV for showers from 0 to 45 degrees zenith angle. This is also the point at which the detector is equally effective for gamma and proton showers, coincidentally. Gamma showers dive off faster than proton showers at lower energies. Similar for Milagro with a 40 tube trigger.

Where does high effective area for proton showers at low energy come from? At low energies, number of particles reaching our altitude is higher for proton showers. But for zenithal showers, this difference goes away or turns over. Also, as you go away from the core center, proton showers become more important. On the pond we have a higher trigger probability for gamma showers, and there is a crossover point at 75 meters or so.

Sensivity to the Crab should be about 0.1 sigma per day (without curvature correction) and 0.15 sigma per day (with curvature correction), and a little higher for Mrk 501 (at 1 Crab flux) because of its higher declination. For a one-year observation of the Crab we therefore expect a significance of approximately 2.3 sigma. This assumes no dead tubes. Things improve somewhat if we don't include events with cores off the pond.

Event rate - take JACEE proton flux, at a 100 tube trigger, get 281 per second in Milagrito, which is roughly consistent with that was seen.

Milagrito Moon Shadow - Morgan Wascko

At our energies, the shadow is noticeably deflected, so we can start doing significant physics with the geomagnetic field. The shadow is still usefulas a tool, to get the angular resolution, and energy response and resolution. With the right simulations, we can test a number of aspects of the reconstruction techniques. We could also detect, possibly, an antimatter component of the cosmic ray flux.

Divide all the moon data into zenith angle bins 5 degrees wide. The shadow seems to get more smeared out closer to zenith because you're looking at lower energy showers, but this is complicated because the shift due to the magnetic field also depends on the zenith angle. Splitting up the theta bands into east versus west halves, there are some interesting features, but the significance becomes rather weaker because of less data. Deflection should be more northerly or southerly in west and east halves, and there is tentative agreement with this.

Solar Event of November 6 - Abe Falcone

Climax neutron monitor saw a huge ground level event with maximum at 11:55 UT, with energies above 3 GeV (protons at the top of the atmosphere). Mt Washington also saw this event at approximately the same time (but integration time was longer), with a lower geomagnetic cutoff. This was an isotropic event, was seen in Switzerland and Moscow.

Milagrito's high threshold scalar rate sees an onset at 12:02 UT and same method yields an onset time for Climax of 12:03 UT (disagreeing with the more complex Climax methods of onset estimation but not significantly). Same duration is also seen, and the 100 PMT trigger also goes up a small amount over the same period. Similarity between the Milagrito curve and the climax curve is also suggestive. Throwing out Patch 7 improves the agreement of the curve with Clmax. Patch 7 has often shown quirky behavior, owing probably to flashers. Apart from this patch, there is good uniformity across the pond. There is a flasher in patch 7 at the same time as the Climax event, but another (or else a light leak) that also occurs in the same patch a couple of hours after the event.

Other flasher events (looking at ~20 events) are highly localized in the pond, confined to a single patch, or at most one other.

High energies should be expected for the Nov 6 event because of ACE/SEPICA data on Fe line suggests very hard spectrum. The response in the 100 PMT trigger suggests there may be some particles even up to 100 GeV in this event!

Solar Event of November 6 - Dave Williams

Low threshold data shows a rise at about 12:00 UT, then bigger rises later as light leaks start to come into play. Looking at individual tubes, the effects are not very visible. Light leaks are obvious, flashers less so. Are there any tubes that have glitches taking place around noon time? The sixteen tubes in a corner of patch 7, edge of patches 2, 8, and corner of 3. These show an effect falling away with distance, strongly suggesting that one of them was a flasher. In tube 103 for example, we see a glitch right at noon, and another half an hour later. This strongly indicates a hardware problem in that area of the pond. Take a fifteen minute interval before 12:00 and fifteen minute interval before 13:00. Compare the difference between these two intervals with the differences between similar intervals separated by an hour throughout the lifetime of Milagrito (989 pairs of intervals). Excluding patch 7 improves the distribution of these differences, eliminating long tails. The difference in the high threshold rate during the Nov 6 event is about 2 kHz, for a fractional increase of 0.00486, with a significance of 1.6 sigma (some question about the nature of the statistics here). There are two points in the distribution that have higher significance. There is a correlation between fractional rate change and pressure differences, but this does not account for the Nov 6 candidate or the other two points. One of the outliers had to do with a recent turn-on of the high voltage, but the other has no easy explanation. Some of the lower points may be related to light leaks.

It looks like the candidate stands out as a rather improbable fluctuation, but there are some other events that may confuse things. We need to look at the overall behavior of flashers in the pond to understand this behavior.

Single, Unaccompanied Hadrons in Milagro - Gaurang Yodh

Unaccompanied hadrons may help to define the cosmic ray proton spectrum. Milagro is the world's largest calorimeter. Single hadron flux samples the surviving proton flux incident at the top of the atmosphere. One in ten thousand 100 GeV protons incident at the top survive down to the altitude of Milagro. Ground level measures (Ellsworth & Yodh, at Sunspot) are consistent with this, although the JACEE absolute flux at 100 GeV is uncertain to a factor of ~3. Other species would probably not be observed, but would make the surviving spectrum flatter.

Single hadrons would make a highly localized cascade in the pond: 5 meter radius in the hadron layer for those that also light up some tubes in the air shower layer. Monte Carlo simulations of such events in Milagrito shows a handful of tubes hammered, with low pulse heights in perhaps eighty tubes.

Data from Milagrito - differential spectrum is -1.7, we probably measure the energy very poorly, flattening the true spectrum. In the real sample, there were 3000 such events in one sub-run. This may be ~2% of our triggers on the pond, i.e. 6 Hz or so in Milagrito. The event rate is consistent with the flux measured earlier at Sunspot.

These are seen easily in Milagrito, and we should therefore be able to make a good physics case using Milagro, which should be able to get some events at higher energies (TeV or so). Milagro needs to have a trigger for events of this kind.

NIM/Astroparticle Physics Article - Jordan Goodman

Large parts of the Milagrito NIM article are yet not done, though some parts were done soon after the last collaboration meeting. Folks are urged to get their parts done, and it shouldn't be too hard to do. This should be a high priority, for political as well as scientific reasons.

We should set ourselves a deadline to finish this. Halloween?

What do we present at the VERITAS workshop?

Andy will present what he talked about in this meeting - Mrk501, Crab results - in a poster. There will be no overview talk about Milagro at the meeting.

Abe & Jim will present a poster about the Milagro solar physics possibilities and the Nov 6 solar event results in tentative language.

Milagro Operations

Discussions -

Monitoring water quality - Do we use something off the shelf, or a homebuilt sysdtem? Do we try multi-wavelengths, or just one? A commercial system uses a small cell, many wavelengths, for about 13 k. The problem with a commercial spectrophotometer is that you have to be very careful about cleaning it, being very careful about performing the measurements, but if you do it well you can get accurate results. The full spectrum is an advantage if you can get it. Michael's home-built system, using a larger cell, costs about 10k for one wavelength, 2k more for two wavelengths. One meter glass cell is the sampling container. Need temperature stability within 5 degrees C during the measurement. Two measurements, empty cell, then full cell - measurement itself takes 10 minutes. This would be done perhaps once a month. Turn it on, let it warm up for an hour, let the bubbles go away, take the measurement, turn it off. Power of laser is few milliwatts, probably class 2. Where would you set it up? Prefer not to carry the water back to town for measurements. Pub is too humid, ops building may be awkward. How about the laser shack? Without some temperature control in that building, temperatures could get to 20 below. How about the office trailer? Another useful item will be the "fish tank" with a laser diode (in the blue only) that will run every day and you can monitor global deterioration or improvement, but is not terribly accurate, especially in the uv where it really matters. This tank is at Maryland, and will be shipped out here for installation in the PUB. Todd and Michael and Dave Williams and Don Coyne (or some of the above) will go to Varian to look at how the commercial system might be modified for our use.

Calibrations - What tasks remain? The 30 laserballs in the pond are untested, but are presumed to work. We have one "laser pointer" with quartz fiber from the main laser, that can run at 337 nm, up on the slope by the entry hole, produces a 3 inch beam spot at 70 feet, and 9 inches across the pond. Focus is not particularly sensitive. This could be used for monitoring the water quality across the entire pond. Two tasks still undone for Milagrito are the recalibration/re-sampling and the absolute energy calibration, though Isabel has made some progress on the first of these. For Milagro, the calibration system needs to be automated, and (secondarily) made remotely operable. Based on Isabel's experience with Milagrito, a calibration run for Milagro may take as long as a day. Remaining to be done: connectorize the quartz fiber, reassemble the laser optics, improve the gas regulation, reassemble the laser controls on the SGI (was on Mac LabView) so it can be run via the DAQ, recalibrate the filter wheel, and put in a monitor for the laser output.

Access to Milagro computers - John O'Donnell

Details in handout from John. All of the Milagro analysis cluster has now been upgraded to use only ssh and not the insecure processes used before. SGI machines are well known to be insecure, and attacks have occurred. ftp does not currently work with ssh; have to use scp to copy files. F-Secure on PC and Mac does not implement scp in Version 1.x. Version 2 is reported to add scp, but John hasn't tested it yet. Identities are also reported to work in Version 2, but not in version 1.

Important - do not telnet to an unsecure machine and then ssh to a secure one because your password will be sniffed on the first leg!!!

Further Discussion

Working Groups - conveners are listed first
Operations (Water, Pond, EMS, Maintenance, Shifts) - Cy, Gus, Don C., Michael, Neil, Andy

Code and Data (Reconstruction, DAQ, Online Analysis, Data Management) - Gus, Jordan, Tony, Mei-Li, Miguel, Andy, Allen, Julie, Rich, Stefan, Joe

Data Management - Andy, David, Miguel, Julie

Simulations - Stefan, Morgan, Joe, Julie

Calibrations - Todd, Peter, Fleysher (2), Cy, Gaurang

Outriggers and Other Upgrades - Tony, Neil, Scott, Don, Jim, Abe, Todd

Solar Physics DAQ - Jim, Abe, Galen, Rich, Dave

Let's have a phone conference every other week among the conveners. Each group should prepare ahead of time a summary of what they've done, this will be posted to the web following the phone conference. For the people who aren't here, the possibility of signing up to at most two of these groups will be made available via the web.

Publications - if it's your thesis topic, the author list should somehow reflect that, either first author or with an asterisk. For the most part, the list should remain alphabetical. How long do people stay on the author list after they leave the collaboration? Jordan will write the guidelines, but each institution will handle its own people. Doing talks - there will be a committee (Gus, Gaurang, Don) that decides on a list of conferences and distributes this to the collaboration. Everyone should have an opportunity to talk about the experiment and the wonderful results that will come from it.

Jordan is talking to Gene Low at NSF next Monday and needs vu-graphs with real physics selling points to buttress our proposal. Several people volunteered to provide these. What we can do that no one else can do is especially important - wide area searches, grbs (measuring cut-off especially), pbh, solar stuff, etc.

How many tapes per day is reasonable? - Experience with Milagrito is slightly more than one tape per day.


When do we start taking data?

Milagro Software

Calibration software, automation - Fleyshers

Have written some software, controllers, etc, to run on daq computer (maya) or on ems computer, rather than on the master computer (kahuna). Check out is in process.

DAQ - Richard Miller

Some small changes need to be made to the Milagrito daq, but these can't be made until data taking starts. The data format is still to be decided.

Data Compression - Miguel Morales

Data compression for the calibrated data, not the raw data. Tube-by-tube data, after edge detection, etc. Space saving techniques - save only the necessary information (nothing that can be re-created, for example, like core positions); do variable word-length compression; bit-by-bit saving. An example of variable word-length compression: our data is very bunched within the pond, and you don't need to save, for every tube, its full-length number, just a delta, with a flag bit. Very highly peaked data can be compressed very efficiently using deltas and flags.


Variable compression scheme for both pe and tchi.

Tchi – using a hyperbolic tangent

Joe –

Monte Carlo

+- 200 m 20m /20m absorption /scattering

baffles on both layers

He threw 135k gamma showers – 13000 triggers (E-2.0)

210,000 p showers 3,875 triggers (E-2.7)

He obtains about 1 degree for milagro when he uses fitted cores and about .6 when he uses true core.


Data Format –

Want fast I/O

Efficient code

Idiot proof.


Widely used and developed at NCSA

Data is stored as objects

Version 4 is fully developed, but is not really designed for huge data sets

Version 5 is not ready yet, but is being aimed at our type of data set.

A function can be used to tell how well its working.On/off pond core separation - Lazar Fleysher

Sensitivity on a source depends on how well you can separate showers with cores on the pond from those with cores off the pond. Simulations of new filters (which attempt to throw out cores off the pond) show reduction of signal to background by a factor of nearly 2, but signal to noise ratio actually gets worse because of great reduction (factor 3) in the signal. That is, 2/3 of good cores were thrown out by these filters. It may be possible to improve these filters, though the path is not yet clear. Chief problem is that the pond is too small, not well matched to the shower size.

Angle finder w/built-in curvature correction - Roman Fleysher

Use a parabolic curvature fitter and angle finder. Need to find core location, which gives two of the six parameters required to fit. The resulting expression is quadratic in the pmt coordinates, which results in a linear fit. This can be performed by inverting a matrix, and could work faster than the existing iterative fit. The timing information in principle contains the core location, which falls out of the fit. But what happens when there is no good parabola in the timing? Then presumably you loose the ability to find the angles as well as the core. Can you fix the curvature parameter and reduce the order of the fit? Apparently not. The algorithm works, but still requires some testing.

What are our priorities - Cy Hoffman leads discussion

We have to get Milagro up and running, but at the same time analyze Milagrito data for presentations (especially important for graduate students). Question - do we continue reconstruction of the old data or do we stop with the reconstruction that we already have? If we accept the existing reconstruction, it is conceivable to do both the analysis of Milagrito data and get Milagro running at the same time. Developing the analysis tools for Milagrito actually helps getting Milagro running; we should probably focus on the things that can be done in common for Milagro data and Milagrito data. Sky maps, for example, should be done and tested for Milagrito data, and then will be ready for implementation on Milagro. In improving things for the Milagrito data, we should focus exclusively on things that will also help Milagro. Outliers - we've committed to putting fourteen of them out at the site before winter, but it seems unlikely that we'll be able to do so. Why fourteen? Only two or three are necessary to verify that the Monte Carlo simulations of the tanks are valid, but you need about fourteen to start finding cores. Should we activate the schmoos to help with our understanding? A big issue is how to write the proposal for the outriggers (etc.) which we need to do before we really have any outrigger data. WACT and the solar physics add-on are semi-independent of the other priorities.

Next collaboration meeting

When should it be? End of January or February may be good because it will be just before the physics runs, we should already know some of the calibration results, software that will be available, modes of operation, etc. By that time we should also have finished analyses on Milagrito. Where should it be? Santa Cruz has volunteered.

Time for conveners' phone conference

Probably a week from Thursday (October 29th) at 10am MST. Jordan will announce.

The End!

Galen Gisler E-mail: Mail Stop D434 Phone: 505 667 1375 Los Alamos National Laboratory sometimes: 505 667 0400 Los Alamos NM 87545 USA Fax: 505 665 4414