Towards a Discipline oF Practice - Evidence Based Medicine 3

In my last post on evidence based medicine, I suggested that professional training focused on the transfer of existing knowledge and skills to the new professional, knowledge and skills that the professional then applied. The professional subsequently built on this base through practice, at the same time using various professional development activities to try to keep in touch with new developments.

I then posed the question what happens if that existing knowledge base is in fact wrong, looking briefly at the reasons why this proved to be the case in medicine, a discovery that had led to the development of evidence based medicine. I concluded that, as with any other approach, evidence based medicine had its own methodological problems. However, it also had important lessons for other fields of professional practice.

The Quality Movement and Quantification

In a series of posts on my personal blog I explored some of the changes that had taken place in public administration since the war, looking at the influences on those changes.

In one of those posts I looked in part at the way in which standards, the Quality Movement and the importance of measurement had become major global influences. I also suggested that the outcomes here had not always been positive.

Evidence based medicine forms part of the global standards and quantification revolution and suffers from some of the same weaknesses. These weaknesses need to be recognised.

Problems with Evidence Based Medicine: Perception Bias

The first problem can be called simply perception bias.

In another post on my personal blog on science and political correctness I looked at ways in which dominant views acted to exclude alternatives.

Evidence based medicine is neither value nor perception free. The questions selected for test and evaluation, a process that can be very expensive, are influenced by prevailing views. Valuable alternatives may be excluded simply because they fall outside conventional wisdom. As evidence based medicine becomes the dominant mode, the effect may, as it has been in other areas, to actually narrow fields of investigation and action.

This links to a second problem, one that I have discussed before, the tendency for all professions to see answers within a frame or world view set by their profession.

A lawyer will give you a legal answer to a problem, a doctor a medical answer. If you have a back problem and see a surgeon, he/she is likely to think about surgical solutions. Go to a chiropractor with the same problem and he/she is likely to recommend spinal manipulation. So professional background helps determine the way the problem is defined, the solution applied.

This flows through into the application of evidence based approaches because the things tested are generally set within the frame of the tester. So evidence based medicine focuses on the efficacy of medical treatment and may leave non-medical alternatives aside.

Problems with Evidence Based Medicine: Causation

As part of my history honours year in my first degree I studied philosophy of history under Ted Tapp. Ted was a reflective man who required us to think about, to debate, the philosophical underpinnings of science and scientific method.

One core problem was the difference between correlation (a and b) as compared to causation (if a then b).

This problem applies in evidence based approaches. Just because a study shows an apparently strong relationship between a treatment and positive patient outcomes does not necessarily say anything about the causal relationship between the two. This has to be deduced and further tested.

Problems with Evidence Based Medicine: Problems of Epidemiological Studies

The problem of correlation vs causation links to another group of problems with evidence based medicine.

By its nature, evidence based medicine deals with large groups, populations.

As trials become larger and more complex, it becomes more difficult in statistical terms to establish significant relationships, to separate the effects of different variables.

This creates another problem, the establishment of a clear relationship between the outcomes of trials at population level and subsequent application at individual level.

As the Wikipedia article notes:

Critics of EBM say lack of evidence and lack of benefit are not the same, and that the more data are pooled and aggregated, the more difficult it is to compare the patients in the studies with the patient in front of the doctor — that is, EBM applies to populations, not necessarily to individuals.

This can create very real difficulties for individual clinicians, leading Tonelli to argue in The limits of evidence-based medicine that:

the knowledge gained from clinical research does not directly answer the primary clinical question of what is best for the patient at hand.

Tonelli concludes that proponents of evidence-based medicine discount the value of clinical experience (source Wikipedia).

Problems with Evidence Based Medicine: Impact of the Observer

Another problem with evidence based medicine, one often seen in all evidence based approaches, is the way the observer affects the observed. This happens at several levels.

The first problem is that the simple act of participation in the trial may have some and not clearly seen impact on individual outcomes. In medicine, this is usually managed by use of a control group using a placebo. The efficacy of the treatment is then measured by the difference in outcomes between the control group and those receiving the treatment.

A second linked problem is the impact on patient behaviour of the trial itself. By their nature, clinical trials are closely managed. This means that patient compliance with the treatment routine is likely to be high.

This need not hold in subsequent clinical use since ordinary patients are more likely to fail to follow treatment processes by, for example, failing to take medication exactly as prescribed. This means that actual patient outcomes may not be as good as the trial results.

Problems with Evidence Based Medicine: Limitations in Application

A further problem is that the most rigorous gold standard approaches dictated by evidence based medicine can only be applied in narrowly defined circumstances, leaving a range of medical approaches that have to be tested by less rigorous means.

This should not matter so long as the limitations are recognised. In practice, it risks introducing two distinct distortions into the medicine and the health system.

The first is the risk that investigation may be biased towards those things that can be measured through more rigorous techniques, reducing thought and investigation in areas less amenable to measurement.

The second related risk is that treatment itself may become biased.

At clinician level, this links back to my earlier point about perception bias. Doctors trained in evidence based medicine may, consciously or unconsciously, come to focus in treatment terms on those things that can be measured, ruling out other less easily measured options.

This tendency may be reinforced by actions from those managing or funding the provision of health care services who may refuse to allow/pay for certain types of services notwithstanding the views of individual clinicians.

Next Post

I have focused in this post on problems associated with evidence based medicine. In my next post I will look at the lessons of evidence based medicine for other professions.

Previous Posts in the "Towards a Discipline of Practice" Series

• 21 September 2006 Towards a Discipline of Practice
• 22 September 2006 Role of the diagnostic in professional practice - medicine vs law
• 8 January 2007 Reflections on Professional Practice and Practices
• 16 January 2007 Towards a Discipline of Practice - Evidence Based Medicine 1
• 22 January 2007 Towards a Discipline of Practice - Evidence Based Medicine 2

Towards a Discipline Of Practice - Evidence Based Medicine 2

Evidence-based medicine (EBM) is the integration of best research evidence with clinical expertise and patient values ... When these three elements are integrated, clinicians and patients form a diagnostic and therapeutic alliance which optimizes clinical outcomes and quality of life. Centre for Evidence Based Medicine

While this quote focuses on medicine, it also captures four key elements relevant to all professions and professionals.

The first element is research evidence, essentially what works and why.

The second element is professional expertise, our capacity to understand and apply our professional knowledge in the circumstances of the particular case.

The third element is the patient or client, each with their specific attitudes and needs.

The final element is the integration of the first three elements - the diagnostic and therapeutic alliance - to provide the solution that best meets client needs.

In my brief introductory post on the case of evidence base medicine, I suggested that my instinctive first reaction on hearing about evidence based medicine was to say that's odd, I thought that all medicine was evidence based. I also suggested that this has proved, in fact, to be far from true.

Before continuing my analysis I should note that the Australian Broadcasting Corporation's Radio National has just completed a rather good two part series - Facing the Evidence - on evidence based medicine. The first part is already available in transcript and pod cast. The second will be available shortly.

Returning to my theme, when you look as I did in my last post at the standard way all professionals are trained, you can see that that training focuses on the transfer of existing knowledge and skills to the new professional, knowledge and skills that that professional then applies. The professional then builds on this base through practice, at the same time using various professional development activities to try to keep in touch with new developments.

But what happens if that existing knowledge base is in fact wrong? How might this arise? To quote from the first part of the ABC program:

Every day doctors and other health professionals use treatments that are harmful, or fail to use therapies that have been proven to work. In the US there is so much medical error that Congress has directed the Institute of Medicine to develop a strategy to improve the quality of care. In its initial report the Institute noted perhaps as many as 100,000 Americans die every year from medical errors, including the use of inappropriate treatments. That's much more than from car accidents, breast cancer or AIDS. Many more suffer side effects and unnecessary costs

How could this happen?

Part of the problem here lies simply in the placebo effect, that fact that patients respond just to the fact of treatment. So the treatment appears to work, thus supporting the original judgment.

Part of the problem also lies in the fact that individual outcomes can be affected by so many variables and over a considerable time horizon so that the fact of adverse outcomes may not be clear in an individual case or, if clear, may be due to a whole variety of factors external to the treatment itself. There is a linked issue here that relates to the size of the population.

Given that individual outcomes vary greatly, the fact that there is a problem and its scale and scope may only become clear if you look at a population as a whole. That is, the individual professional may have no easy way of detecting the problem in his/her individual cases.

A further problem lies in the nature of the models used.

All professions use models to try to explain a complex world. In economics, for example, models are a common method used to analyse economic behaviour and to suggest possible responses at firm and public policy levels. In the case of medicine, biological models are common.

The problem with all models is that they involve selection of key variables and the specification of relations between those variables. Get either wrong, and outcomes may be very different from those projected by the model.

Doctors have always been concerned about adverse or unexpected outcomes.

In 1972 Professor Archie Cochrane, a Scottish epidemiologist published Effectiveness and Efficiency: Random Reflections on Health Services. This plus Cochrane's subsequent advocacy caused increasing acceptance of the concepts behind evidence-based practice. Cochrane's work was honoured through the naming of centres of evidence-based medical research — Cochrane Centres — and an international organisation, the Cochrane Collaboration.

The explicit methodologies used to determine "best evidence" were then largely established by the McMaster University research group led by David Sackett and Gordon Guyatt. According to the Wikipedia article on evidence based medicine, the term "evidence-based medicine" itself first appeared in the medical literature in 1992 in a paper by Guyatt et al. (Guyatt G, Cairns J, Churchill D, et al. [‘Evidence-Based Medicine Working Group’] "Evidence-based medicine. A new approach to teaching the practice of medicine." JAMA 1992;268:2420-5. PMID 1404801)

From this point, the spread of the concept and its subsequent inclusion in professional training was rapid.

As with any other approach, evidence based medicine has its own methodological problems. However, it also has important lessons for other fields of professional practice.

Previous Posts in the "Towards a Discipline of Practice" Series

• 21 September 2006 Towards a Discipline of Practice
• 22 September 2006 Role of the diagnostic in professional practice - medicine vs law
• 8 January 2007 Reflections on Professional Practice and Practices
• 16 January 2007 Towards a Discipline of Practice - Evidence Based Medicine 1

Towards a Discipline of Practice - Ndarala Case Study

One of the reasons why I and my colleagues are so interested in concepts such as a discipline of practice are the challenges we have faced in trying to build Ndarala as a multi-profession network of independent management related professionals and professional practices.

For that reason I thought that it might be of interest if I posted a story on our experiences on the Ndarala Group blog as a case study. While I think that we have some some pretty good things, the story also provides a frank assessment of some of the difficulties we have faced.

Towards a Discipline of Practice - Evidence Based Medicine 1

Evidence-based medicine (EBM) is an attempt to more uniformly apply the standards of evidence gained from the scientific method, to certain aspects of medical practice. Specifically, EBM seeks to apply judgements about the inductive quality of evidence, to those aspects of medicine which depend on rational assessments of risks and benefits of treatments (including lack of treatment). According to the Centre for Evidence-Based Medicine,"Evidence-based medicine is the conscientious, explicit and judicious use of current best evidence in making decisions about the care of individual patients."[1] Cited from Wikpedia.

In this post I want to continue my discussion on the development of a discipline of practice, a discipline focused on the way professionals practice their profession, by introducing in a preliminary way the case of evidence based medicine.

I first came across the concept in 1998 when I started as CEO of the Royal Australian (now Australian and New Zealand) College of Ophthalmologists. My instinctive first reaction was to say that's odd, I thought that all medicine was evidence based. In fact, that's far from true.

To understand this, we need to look at the way in which doctors are trained, as well as the way in which new medical approaches develop.

As with all professions, training starts with the previous body of knowledge relevant to that practice. For practical reasons, much of this has to be taken for granted. The trainee professional simply has to learn those elements required to begin practice.

The trainee then has to learn to apply that knowledge in practice. In the case of medicine this is done especially in hospitals working under the supervision of a qualified doctor who passes his/her knowledge and experience onto the trainee while checking their application. Again, the trainee is acquiring the current wisdom.

These broad processes continue throughout a professional training that can, in the case of medical specialists, extend as long twelve or thirteen years.

Once the doctor begins practice, he/she continues to learn from experience with patients. Doctors also learn through contact with other doctors and are expected to maintain a program of continuing professional education to keep them in touch with latest developments. Similar approaches and programs apply in other professional areas.

None of this will seem strange to any professional from any discipline. Yet in the case of medicine the process has proved to be seriously flawed. Evidence based medicine attempts to address these flaws.

Previous Posts in this Series

• 21 September 2006 Towards a Discipline of Practice
• 22 September 2006 Role of the diagnostic in professional practice - medicine vs law
• 8 January 2007 Reflections on Professional Practice and Practices

Teleworking - a Questionnaire

Several years ago I made a personal decision to work mainly from home. I did so for family reasons because I wanted to play a more active role with my girls while they were growing up. There have been costs, but also enormous personal gains.

Last November I wrote a story on the Ndarala Group blog looking at some of the issues involved in teleworking from a personal perspective. I have now been approached by Andrew Buckland seeking help on his Masters thesis.

Andrew is reading an MSc in Facilities and Asset Management at Edinburgh's Heriot-Watt University. His related dissertation centres on establishing if home working is a viable alternative to the modern office environment. As part of this, he has developed a short questionnaire to gather information from people who have experienced working from home. His focus is on those who have worked from home but who are not self-employed.

I know that a number of people in professional services do work in whole or part from home, a trend that has been increasing in part because of the steady increase in the proportion of women in the professional work force. Given this, I thought that some of my readers might be able to help Andrew by filling out the questionnaire. He can be contacted via email - andrew@ascb.eclipse.co.uk.

For the benefit of my international readers, the Australian Government's Australian Telework Advisory Committee (ATAC) has been investigating the teleworking option. The Committee's report plus supporting reports and submissions (here) contains a range of useful information on teleworking.

Offering advice when it's not been asked for

One problem that many professionals have experienced - I know I have - is just what to do when you see a client that has a real problem, but one that falls outside the scope of your immediate work.

What to do? You want to help the client, but if you are to do anything you have to find a way of intervening. Too often, and again I speak from experience, you end up alienating the client without achieving positive results.

I mention this because David Maister's Passion, People and Passions has an interesting and useful discussion thread on this topic. I won't repeat the arguments, but refer you to it including the comments made.

Reflections on Professional Practice and Practices

This Al Jazeera International photo shows a helicopter water bombing during last year's Australian bushfires. The point of the photo is that once you have to start putting out fires from on-high you are already in trouble. It is generally better to deal with the fire at source while it is small, and that comes back to improved management.

I have been musing about issues associated with professional practice, trying to clarify and structure some confusions in my own thinking. One of my key concerns has been a feeling that I need to find a better way to focus, link and present my own writing and thinking to make it more accessible.

Three Overlapping Knowledge Domains

In an earlier post, Towards a Discipline of Practice, I suggested that there were three overlapping knowledge domains within professional services:

1. The profession itself, whether it be engineering, medicine, law or training. What we do. Most professional education and training focuses on this.

2. The application of the profession in practice by individual professionals. Doctor/patient, lawyer/client etc. Essentially, how do we do what we do.

3. The management of the overall practice. Essentially, how do we manage what we do. I am using the term practice in the broad sense to cover the variety of business structural arrangements found within professional services.

We can think of these domains in terms of a matrix with the three knowledge domains and their various subdivisions down one axis, the various professional services fields and their subdivisions along the other.

The Challenge

When I began this blog my focus was, as the blog name suggests, on number 3, the management of the professional services practice. This remains my core focus. However, I have in fact found myself writing across all three knowledge domains.

The problem with this is that it can create confusion in the minds of individual readers. To tease this out a little, hopefully encouraging debate, I am now going to look briefly at the first two knowledge domains.

Knowledge Domain One: the Individual Profession, Knowledge Domain Two: Application of the Profession in Practice

While these two areas are very distinct, I have linked them together because collectively they bear upon one of my key interests, ways of enhancing cooperation between professions to facilitate true multidisciplinary working. This links to a second interest, the extent to which professions can learn from each other.

Originally, there were arguably three great professions, law, medicine and theology.

One of the features of the process of professionalisation was the way each developed its own knowledge domain, creating a separation and mystique from other fields of human activity and knowledge. Linked to this was the creation of self-governing processes giving the profession the capacity to define and recognise its own practitioners. As other professions emerged, they attempted to follow the same route, creating a professionalisation process.

In saying this, I do not want to become involved in the debate as to what constitutes a true profession. To me, the key distinguishing feature here is that a profession has a focus on the work of the individual professional. Rather, my concern is on the way professions and professionalisation create intellectual and cultural divides that limit cooperation and knowledge transfer between professions.

To some degree these divides are both inevitable and necessary. Every profession needs its own language, its own defined areas of knowledge, to facilitate development of and work within that profession. But problems arises when the divides are such that they blind professions (and professionals) to the gains that can arise from cooperation and cross-fertilisation. Problems can also arise for professions and professionals where professional barriers act to prevent the profession responding too needs that they were created to serve.

Let me try to illustrate all this by an Australian example.

Ophthalmology and Optometry

In Australia as in many other countries, there have been concerted efforts at official level to break down competitive barriers previously protecting the positions of certain professions. These efforts have been opposed by some professions under perceived threat, at times supported by professions who saw potential gains.

As part of this process, there were moves in different Australian states to grant optometrists prescribing rights for certain drugs, thus broadening the range of optometric practice. These moves were opposed by many ophthalmologists and by the Royal Australian (now Australian and New Zealand) College of Ophthalmologists on the grounds that optometrists lacked the knowledge, skills and judgement required to diagnose, prescribe and treat safely.

Despite these objections, the Victorian Government decided to grant certain certain prescribing rights and then asked the College to specify what additional training should be required before individual optometrists could exercise those rights.

The outcome here is described in a case study on the Ndarala Group blog. To provide the necessary advice, the College had to specify just what knowledge and skills were required to diagnose and prescribe safely and effectively so that this could be embodied in training activities. To do this it adopted a competency based approach, applying knowledge and skills drawn from the training profession. In turn, this has fed on into major changes in the approach adopted to the training of ophthalmologists in Australia and New Zealand.

The Lessons

All this may sound a bit arcane, but if we look at the case it does have broader relevance and not just to Australia.

To begin with, this project combined knowledge domain one (what we do) and two (how we do it). The College actually had to specify key elements in both and show how they were related.

This brings me to my second point. In doing this, the College laid down a methodology capable of application across the professions, one with the potential to demystify professions and, in so doing, to lay the basis for broader cooperation while also opening individual professions up to greater competition.

This may sound extreme, but consider this.

Both Canada and Australia face a problem in regional areas because of an aging lawyer population. Young lawyers are reluctant to take on country general practice positions, preferring instead to look to bigger city specialisations. I may argue that this does not make sense in financial terms if you look at average financial returns, but it is (I think) a practical reality.

Track forward. If certain areas cannot get conventional legal services, then other ways will have to be found to give them that service.

One way of doing this, one that is already happening with conveyancing, is to take out chunks from legal services and allow others to provide the services involved. Another alternative, again one that is already happening, is to move to on-line delivery of certain legal services. These two could, of course, be combined.

Add in the emergence of both corporatised law firms and multi-profession firms including lawyers. At what point do we move to listed entities combining a variety of delivery modes and professional areas?

The initial move to float Integrated Legal Holdings as Australia's first listed law firm ran into a variety of problems including opposition from current legal regulators, although the float is likely to proceed in February. However, this is just a first straw in the wind.

My feeling is that the dynamics already in place guarantee major change over the next few years. I also think that within this change process, the capacity to define knowledge domains in the approach pioneered by the ophthalmology case is going to become increasingly important.

Guidelines for Effective Negotiation

I have been editing some new material for inclusion in the project management for the professional series running on the Ndarala Group blog. This will start coming up shortly. In the meantime, I thought that one item was worth running here as a stand-alone check list.

Ten guidelines for effective negotiation:

1. Be prepared - know what outcome you want and why.

2. Minimise perceptual differences - ask questions to gain understanding.

3. Listen - questions are no good unless you listen to the answers.

4. Take notes - what has been agreed and what remains to be resolved.

5. Be creative - look for new solutions.

6. Help the other party.

7. Make trade-offs - trade what is cheap to you but valuable to the other party for what is valuable to you but cheap to the other party.

8. Be quick to apologise.

9. Avoid ultimatums.

10. Set realistic deadlines.

What Lawyers (and other professionals) appreciate

Back in December Tony Karrer tagged me to join in Five Things Meme requiring me to say five things about myself that other people did not know. Wandering around the blogosphere I was surprised at just how far this particular process spread.

In December there was another tag game, this time centred on what lawyers appreciate. A list of posts on the topic can be found on Life at the Bar.

In his response, Bruce MacEwen (Adam Smith Esq) said in part:

Lawyers appreciate professional management at senior executive levels of their firms. Lawyers are not taught, and by and large don't care to learn about:

• competitive strategy
• management 101
• finance
• marketing
• IT, or
• human resources.

Ergo those functions should be left strictly under adult supervision. Hire worldly-wise and savvy strategic advisers, Chief Operating Officers, Chief Financial Officers, Chief Marketing Officers, CIO's, and heads of HR, and get the lawyers out of their way.

What lawyers do care about is professional excellence, a collegial and fulfilling atmosphere, and above all else the ability to serve appreciative clients with impeccable legal counsel. Lawyers appreciate being able to focus on that to the exclusion of all else. Let them.

While I understand Bruce's point - you only have to look at certain managing partners to see what he means - I also found it depressing.

One of the features of all good professionals regardless of field is that they have, to use one of David Maister's favourite words, a passion for their professions. This holds whether you are a school teacher, a doctor or a lawyer. In this sense, part of the role of management is to help the professional do her or his job, to provide support and then get out of the way.

There is in fact a growing problem here in many professions - teaching is an example - where externally imposed requirements are forcing a growing administrative load on professionals. But beyond this, there are two main problems with the way Bruce phrased his comments.

The first is that the discipline of practice requires professionals to learn certain management skills.

Increasingly, professional practice involves project and team based work. We speak of team teaching, of multidisciplinary working, of teams of health professionals. This has always been a feature of areas such as IT, but is becoming increasingly widespread.

The discipline of practice also requires professionals to manage their relations with clients. The age of the God professional in which the professional dictated, the client accepted, is now long gone.

Increasingly, too, the growth of larger professional services organisations means that professionals are placed in management or staff supervisory roles as part of their normal professional role.

These things require all professionals to acquire at least those management skills relevant to their particular professional circumstances. This can create real problems because, as I have argued previously, professionals are trained in very different ways from managers.

The second problem is that many professionals simply don't have the luxury of following the course Bruce suggests.

If you look across professional services, the reality is that most professional practices are just too small to afford complete professional management. Willy nilly, many professionals including lawyers have to become involved in management whether they like it or not. And many do not like!

Professions and professionals respond to this challenge in different ways.

In the case of doctors, for example, where individual practice had been the norm, multi-doctor practices became common. Each doctor remained in individual practice but with a service company providing shared services (premises, receptionist, accounts etc). Now, with increasing compliance loads, many doctors are opting out of individual practice into paid employment.

In other cases many professionals opt for self-employed status to minimise management load, working on their own or with a single part or full time staff member. This group faces very particular problems in that they either have to learn to do themselves or outsource, and outsourcing has its own problems. In Australia, the present biggest single complaint among self-employed professionals is their inability to get adequate accounting or book keeping support.

One of the reasons for the spread of franchise style arrangements in professional services is that they can help overcome these types of problems.

Those who wish to grow their practices as businesses face a new set of challenges because they often have to learn to manage as they go along. Generally, what these professionals need are basic management and associated business skills, together with simple scalable solutions that can grow with the business. Much of the discussion in the management literature with its focus on the bigger end of town is simply too complicated to really help this group.

My particular passion as a professional lies in finding ways to improve management. In this context I have a particular sympathy with this last group simply because I have been there.

Bridging Professional Divides - Education & Training

Happy new year to all.

I have often commented on this blog about the divides within and between professions.

One such divide that I have noticed is that between education and training professionals whose focus is on the education sector (schools, universities) and those who primarily have a training or learning and development focus external to the education and training sector. Obviously there is overlap, but the divide is still there.

I was reminded of this over the break by a discussion with Doug Belshaw.

For those who on the learning and development side who do not know Doug, he is a UK history and IT teacher who has, among other things, been pioneering the application of new technology in school teaching.

Doug's main blog is teaching.mrbelshaw.co.uk. I commend the blog and especially the post on the 20 best edublog posts of 2006. I noticed that it has listed a post by Jay Cross on informal learning, in fact an example of overlap. Doug also has a blog dealing with topics relating to his doctorate.

Bronwyn Clarke's blog also provides an entry point for training professionals into the parallel education world. Bronwyn is not posting actively at the moment, but has a number of interesting links.

On the other side of the ledger, I said to Doug that I would put up some information that might act as an introduction for professionals from the education sector to the parallel learning and development word with a special on-line focus.

The material that follows simply provides a taste. Those who follow this blog will recognise many of the names.

1. Learning Circuits Blog. This e-learning blog sponsored by the American Society for Training and Development provides a useful entry point.

2. eLearning Technology. Tony Karrer's blog provides a very useful intro to a variety of issues in the on-line training environment.

3. Stephen's Web. Stephen Downes' site contains a range of useful material.

4. Jay Cross has two main blogs - Informal Learning and the Internet Time blog.

5. I find Dave Lee's eelearning because he often comes at issues from a different direction.

6. Brent Schlenker's corporate e-learning often provides insights especially on technical issues associated with e-learning.

7. Dennis McDonald's blog on Managing & Living with Technology remains a valuable source of information on developments in the on-line environment.