Confidential - do not disseminate or link to this document
For reference, see the original sumissions
Here is each question in the evaluation form and the responses we got from the reviewers for each institution.
Excellent; Excellent; Very good; Very Good; Very Good; Fair
brings all the components together. However the ISTS work is just beginning and looks very broad if thin.
Limited research capacities demonstrated; Research agenda not specified
Excellent; Very good; Good; Good; Good; Fair;
Strong partnerships, diverse interests, good acknowledgement of realities. Not clear that the proposal has a clear focus and aggressive leadership.
Very good; Very Good; Good; Good; Fair; Fair;
A loose consortium that will rebroker issues. Good expertise in pockets but little focus or synergy.
Fair; Fair; Fair; Poor; Poor; Poor;
essentially a model deployment rather than a research support facility
Excellent; Excellent; Excellent; Very Good; Good; Fair;
Good alignment of interests and expertise. Clear leadership. Perhaps overcommitment of leadership
Excellent; Very Good; Very good; Very good; Good to very good; Fair
Proven track record, strong leadership. Uncertain involvement in campus IT.
Good; Fair to good; Fair; Fair; Fair; Poor.
Very tight focus on only one area trust management, but excellent skills in that area.
Excellent; Mixed - between good and excellent; Very good; Very good; Very good; Good;
Policy language focus is good.
They have done substantial work in directory building, Kerberos authentication and authorization, and IDAP (although I am not familiar with IDAP). They did note that commercial groupware applications require directories or PKI or both – so that some pressure to implement PKI might be to swat away such a requirement rather than attempt to achieve some security goal.
Yes
Their grasp of authorization certificates (from what they wrote) appears to be good – better than that of the other respondents. Like everyone else, they fail to mention the severe problem of doing secure administration of authorization.
The Computer Services team has already deployed a limited PKI and developed a distributed authentication and authorization protocol for information sharing among multiple campuses. Applications include a PKI-enabled HR/Payroll function using digital signatures. The resumes of the investigators contain significant experience in computer security and PKI.
Some in ISTS. Deployed a limited PKI operating Kerberos infrastructure and rights management. Implemented a distributed authentication and authorization protocol (IDAP).
ISTS has
Partially – through Yvo’s work in threshold cryptography.
Lack of detailed description of work related to PKI. Appear to have some theoretical expertise. Two professors are part of Internet2 Security Working Group
Professor Dsemedt has been involved in the research aspects of PKI for many years and appears to have a good background in the field. PKI and information security is a growing research area at FSU. Several papers by the researchers are listed in the proposal.
No.
Very limited staffing; two researchers specified.
Several of the people proposed for the I2 PKI Labs project have significant experience in both the research and practical deployment issues associated with PKI. I really like the multi-discipline approach to their GIIA center. Bringing the technical, legal, and medical deployment aspects of PKI together should yield excellent results and much good information for higher education.
Their existing experience seems to be strongly PKI-vendor based (probably inherited from federal government sources), with a strong focus on directories. They do not appear to consider the usability problems of large directories.
Dorothy Denning is known expert in cryptography and information security, and Michael Getts developed and deployed server-based PKI. Have experience in implementation and deployment of directory based services. Also, experimenting with digital signatures integration for campus-wide deployment of PKI.
Yes
Denning a security expert, though not necessarily in PKI.
Several Universities as part of the SURA have PKI initiatives. PKI lab at Georgia Tech Research Institute (PKIL) participates in the demonstration and testing activities of the Federal PKI initiative (Electronic Messaging Association’s) Challenge effort, is primary system designer and integrator of SWBSADIS-CSI - a secure information sharing system for law enforcement agencies that employs integrated commercial PKI applications and products, and working on State of Georgia PKI initiatives for government-to-government and government-to-public applications. Georgia State University is implementing enterprise wide directory solution and working to implement a PKI solution to support eUniversity. University of Alabama is part of Net@Edu PKI working group.
Some groups, notably GTRI, have expertise but others may not.
Some of the technical PIs have significant experience in PKI and PKI deployments. This includes the project director for the Georgia state PKI deployment and the creation of the Georgia Tech Research Institute (GTRI) PKI Lab.
They cite a number of activities, showing expertise in PKI. However, it is all apparently X.509 and devoted to engineering of those certificates and their machinery. They don’t show any evidence of having addressed the harder problems – the ones that are likely to make an ID certificate structure crumble.
Yes.
Central directory-based authentication and authorization model is being developed. No PKI related expertise was mentioned.
Maybe
Plans for implementing PKI projects are discussed but no clear prior PKI implementations or research were described.
They appear to have some experience, but no research in PKI. Like many others, they appear to believe that a unified, large directory is right and proper.
None shown
Butler has PKI experience. Campbell has expertise in many areas, may be overcommitted with NSA work.
Clearly
Most
The three PIs have a wealth of experience in computing and computer security. In particular the work on SESAME, the Alliance PKI work, and the campus PKI infrastructure activities are all represented in the proposal. Further proof of expertise in the general field is in their Center for Excellence in Information Assurance. The PI's experience with the Grid infrastructure indicates a demonstrated ability to manage large-scale projects ? like some of the scalability PKI experiments across I2 that could result from this research.
They show some expertise, both in new certificate definition and in the surrounding machinery.
Has expertise with Kerberos, use of asymmetric cryptography, and development of a Java based PKI.
Role-based access control is subject for discussion.
CITI is quite strong. Good experience in basic technologies. Oriented towards use in apps, which is appealing.
Their KX509 effort shows expertise. This may not be tied to new research in PKI itself, but rather in an engineering need to marry two technologies that appear at first to be antagonistic to one another. However, they have doubtless acquired experience in the process.
The PI and members of the CITI staff have developed and deployed a wide range of security mechanisms. Applicable projects include their KX509 effort to use PKI to deploy secure web services, significant work with smart cards, secure proxy authentication, secure video conferencing, timestamping for digital signatures, and other activities. CITI is also committed to open-source PKI.
Expertise in PKI related with PGP, junk keys, and short term certificates. Much of the expertise with related technologies (SMART cards, kerberos, GSSAPI)
Excellent expertise in Keynote and authz issues.
The two PIs have excellent technical credentials in computing, information security, and PKI. Their previous work demonstrates a clear understanding of the issues and how to advance the state of the art in the area. Their team has made other important contributions to network security.
Have some experience in related areas. It mostly relates to trust management system, called KeyNote. Also some experience relating to work done on IPSEC, and ISAKMP.
Yes, in their KeyNote work. However, it is parochial – single-focused on that work.
Some
The PIs and staff have years of experience in PKI technology, deployments, and research. Examples include the PKI middleware included Condor, the PKI work being done by the Wisconsin IT staff, and in the background of the principal staff.
Clear record of previous accomplishment and interest specific to PKI.
Performance metrics and policy languages important areas.
Condor project has 18 Months Experience in integration and deployment of PKI technology. Have professors, and staff with expertise in Crypto algorithms, directories, authentication and authorization, etc.
They show some. Their KeyNote experimentation is far better than X.509 experimentation, since it at least breaks free from the path laid out by commercial CA interests.
The proposal to build a secure, usable browser is a welcome initiative, not shared by the other respondents. This is not an easy task, but a necessary one if the user is to be included as part of a security decision.
A focus is the examination of various authorization techniques for use with a PKI. Issues such as the use of authorization certificates vs. authentication certificates and other sources of authorization data will be examined. Another focus area is with secure cryptography coprocessors. These coprocessors offer some potentially interesting solutions for some of the privacy, scalability, trust model, and legal issues associated with PKI.
Finally, they also indicate a plan to work on browser technical and human interface issues and, in the longer term, deal with some of the issues associated with what they term “pervasive computing”and how this breaks many existing security models.
Authorization, access rights allocation and delegation of trust; evaluation for inter-operability and scalability; certificate management among servers. Integrating IBM secure coprocesor with PKI; privacy; augment a browser with ability to authenticate identity of who they're connected to; and PKI for machines.
Also, would like to pursue issues related to pervasive computing, and secure co-processing.
A pretty broad agenda for ISTS. Some, such as secure coprocessors, seems tangential. Some seems a bit naďve. Authorization work seems most interesting
The primary research topics discussed were issues related to attacks against hierarchical CAs and the tradeoffs in expense to build more secure multi-connected PKIs. It was difficult to map the proposal to the I2 PKI Labs Draft Research Agenda.
No evidence
Nothing new or especially interesting is proposed here.
None
Yes, have very specific proposal. Investigation of alternatives for CRL, cost effective PKI, attribute certificates in authorization area, secure storage of keys on servers, recovery of information, registration, user support and training, etc.
Georgetown is the only respondent to propose following the Federal Bridge CA approach. I personally believe this is exactly the wrong thing to do -=- in that it enlarges the name space, making ID certificates that much less useful – but it might be interesting to encourage someone to try it and find out whether it works. The danger here is that from their close working relationship with federal PKI efforts they might find it difficult to criticize such a fundamental piece of the federal PKI plan.
Excellent. The Georgetown proposal includes a research agenda that was clearly focused on the research and deployment issues that will be most critical to higher education over the next few years. Their research agenda demonstrates a clear understanding of the major issues and proposes to work towards solutions in a logical, organized, and cooperative way. The proposed agenda directly corresponds to many of the critical Draft Research Agenda items in the I2 request for proposals.
Research agenda seems more broad than deep. Wanders through a lot of territory, but without a specific focus. Human interface addressed, although on a limited basis.
In general SURA proposes to address the Internet2 agenda, including interoperable policy strategy and implementation.
There was nothing that stood out as a new or interesting extra research proposal. PKI was treated more as an object to be implemented than a topic to study.
The proposal states that the SURA PKI Consortium?s research agenda fits well with the I2 PKI Labs research agenda. However, I was unable to locate specific information in the SURA proposal that went into technical detail on this issue. The section from the GTRI PKI Lab mentioned many capabilities but did not focus on the specific plans or deliverables for the I2 PKI Labs project.
Unclear. Will remarket issues to groups within SURA. Seems more operational than research.
Addresses some of the issues: Directory oriented SSO, VPN, privacy and distributed management .
The proposed areas of PKI Research on interoperability for multiple applications and services are important. However, the proposal did not contain specific technical details on how some of the described issues such as automated issuance of certificates or privacy would be examined. I found it difficult to map their proposed research activities with the I2 PKI Labs Draft Research Agenda.
They appear to consider only implementing the current state of the art, rather than advancing it.
Minor areas
The specific proposed research areas for the PKI Lab were a little vague when related to the goals of the PKI Labs solicitation. However, the general stated concepts of a more interoperable, agile, and scalable PKI infrastructure to meet the demands of grids and advanced applications certainly hit the mark in a general sense. They offer to provide a detailed project plan which, I expect, would go more directly into some of the looming technical issues for large-scale PKI deployments specified in the I2 PKI Labs Draft Research Agenda.
Not clear. Extending Kerberos with PKI, and integration with CORBA are mentioned.
Sesame work combines the technologies growing in importance and penetration, Kerberos and PK and Java.
The active-capabilities effort of the Seraphim project looks like an interesting avenue for study. I wonder if they have investigated PolicyMaker for that effort.
The delegation of privileges in SESAME via PACs shows some inventiveness. It would be interesting to see work to integrate PACs with SPKI authorization certificates, X.509 attribute certificates and perhaps KeyNote certificates.
Seems a bit more oriented towards active security in networks than specific PKI work. SESAME seems relevant
Their marriage of public key authentication to AFS, with authorization derived from AFS, sounds like an interesting approach. I doubt that it generalizes, but it might and could then provide a new mechanism for doing authorization.
Their existing KX509 project is directly related to the I2 PKI Labs goals of harmonizing PKI with existing single sign-on infrastructures. Their efforts in PKIX Time Stamping and short life certificates could have a direct bearing on the I2 PKI Labs CRL agenda. Experience integrating PK-based technology into various applications also exists.
Unclear
Some, short term certificates as replacement for CRL. Experimentation with junk keys for distributed delegation. Also, using short term certificates for bootstrapping long term certificates
Research agenda is right on target. Covers mobility, scaling, authz, etc.
The description of trust management and how the KeyNote system functions is appealing given the large and decentralized nature of Internet2. Their proposed research agenda includes important topics such as delegation, revocation, credential discovery, and automated credential issuance and renewal.
Specific research tasks tied to the I2 PKI Labs Draft Research Agenda include their approach to authorization, handling large scale PKIs, and credential distribution and revocation.
Covers just authz
Nothing especially new or interesting was proposed.
Some
Some, will focus on delegation, revocation, automated issuance, and distribution.
Their plan to consider a prolog-like language (or use Prolog itself) for authorization has real promise and is new enough to be strongly encouraged.
It would be especially interesting to see a study of how to combine security information (ID, attribute or authorization certificates – from X.509, SPKI, KeyNote, PGP, etc.) with that presented in Prolog to arrive at a security decision.
Their overall focus will be on the basic algorithms underlying public key cryptography and working on a security/authorization language.
Specific issues in the proposal that match the I2 PKI Labs Draft Research Agenda include large scale PKI performance issues, policy definition languages, policy bridge CA mapping work, automated generation of certificates, proxy certificates, and other issues.
Condor leverage modest. Ability to maintain task and achieve agenda uncertain too limited resources.
Yes, clearly defines deliverables. Among them are wide area deployment of PKI, integration into middleware, automated generation of certificates, authorization policies into certificates, lightweight certificate authority, performance metrics, interoperability between very different organizations (i.e government and higher ed), certificate proxies, etc.
Unclear
Yes!
Have an existing institutive (ISTS) dedicated to various aspects of security that is funded at $15M/year, however it is not clear as to what resources can be leveraged. They also collaborate with a dozen other leading universities, and research labs in and out of government.
ISTS is new but has some good support. Campus commitment is there. Corporate involvement seems goo.
Yes, ISTS with an existing $15M funding.
Yes, a big one.
Dartmouth will be able to leverage the well-funded Institute for Security Technology Studies. The institute will help to ensure the success of a Dartmouth I2 PKI Labs and help to provide national visibility to the activity.
The FSU PKI group has received $125,000 funding from their College of Arts and Sciences to set up an information security lab. It is not clear how much of the lab will be dedicated to PKI activities but they are receiving backing from their school for this type of work.
Received 125K in grant to set up information security lab
The current $125,000 grant and 2 existing faculty seems small compared to other respondents.
Modest
No
GIIA mentioned; other units bring interest but not necessarily resources.
Their relationship with the GUMC could provide them with a user community and real demands to be met. The relationship with the Health Privacy Project could provide additional valuable guidance in the form of requirements.
They propose to make their PKI work part of the Georgetown Institute for Information Assurance. This relationship would leverage the existing assets of the Institute and help the PKI Lab activity move forward quickly.
Some
Would make it part of GIIA, but does not mention specific funding sources or other resources.
SURA is prepared to contribute $100K over life of the project, and leverage PKIL. SURA PKI consortium is also being proposed.
Yes, the GTRIPKI lab is a direct existing resource. In addition, by being a consortium of various universities, SURA has the potential of a larger than usual user base and a more diverse than usual set of requirements.
Excellent: SURA will provide up to $100,000 in matching funds for the PKI Labs.
Establish a SURA PKI Consortium of seven schools for participation in the project and testbed activities. Successful testbeds could be expanded to the larger SURA community.
The I2 PKI Lab itself would be established at the Georgia Tech Research Institute PKI Lab.
Unclear
Only in a distributed fashion
There is a tie-in with their ARSC, but because it is DoD-funded, they are likely to be forced to stay in the X.509 camp and avoid some of the more interesting topics.
Good use of other resources
None
Not Much
Staff from the Arctic Regions Supercomputing Center will work with University of Alaska staff on this project.
Yes!
NCSA and Allience partners are interested, and will devote 50% of a senior developer to work on PKI. They will also leverage other grants if proposal s successful.
Significant leverage is provided in this proposal. At least 50% of a senior developer will be dedicated to PKI activities along with other resources from NCSA and the campus PKI activities. Their existing research activities fit well into the needs of a PKI effort and these established programs should help to ensure a stream of graduate students to work on various aspects of the project.
Have Grid and NSA connections.
They have several existing activities, but it is not clear that the PKI Labs would be tightly integrated with those efforts.
The CITI organization is well funded and could provide a good organizational infrastructure for the I2 PKI Labs. The proposal calls for the contribution of significant staff time to the I2 PKI Labs project.
Yes
Has some related projects like KX509. A researcher is funded through ITD. Other funding sources are not clear.
Less clear. Some existing commitments from industry, but largely soft money.
CITI has a good track record, but the response does not spell out any other resources or relationships that will be leveraged.
DSL has some resources. Little human interface expertise
Not clear from the proposal.
This proposal leverages the existing equipment and staff at the Distributed Systems Laboratory. Other grant funding for KeyNote/Strongman will also help to ensure a critical mass of staff to work on the issues. Their existing collaborations with AT&T Labs, especially given AT&T?s donations to the PKI Labs effort, could be a great tie in.
There is some other work already underway, but no major institutional tie-ins.
Not much
Excellent coupling of IT org and Condor group.
Their proposal leverages their work on Condor and its association with NCSA and the existing middleware group.
University of Wisconsin is allocating some money (around $90k per year). Also, Condor project is funded at $1M a year from various sources. They will consolidate all PKI efforts with this grant
The Condor project could provide a good test bed for experimentation.
Unclear
Excellent ties to Early Adopters and to campus IT.
Yes, the proposal is a joint effort by the Computing Services group, the Computer Science department, and their Institute for Security Technology Studies.
Yes
Relationship with University’s Computing Services group that has deployed a limited PKI.
They list Dartmouth’s directory of technical services as an investigator. This is not proof that the working relationship is close, but it is a good sign.
Implicit, not explicit.
There is no direct evidence of planned work with their campus computing infrastructure people. Their CIO is active in Internet2 and the two principal faculty members are participating in the Internet2 Security Group.
Cited, but no real evidence of support and interaction.
That working relationship was not spelled out in the response.
Not mentioned in the proposal.
No
Exceptional. Very tight integration. May lead to a too short time horizon.
The Georgetown proposal is a joint effort by academic groups and central IT staff. A close working relationship is evident from their proposal.
Not mentioned, but sounds like they will work with UIS (internal IT org)
yes
The ties to their UIS are spelled out in their response.
SURA is a consortium of forty-seven universities in 13 southeastern states and the District of Columbia. Presumably, they can provide a very good test bed by linking current research with operational deployments within the participating institutions.
The project is composed of groups from central computing organizations including CIOs and staff members and from the GTRI PKI Labs. The plan is to engage SURA academic researchers on the I2 PKI Labs agenda while ensuring coordination with central computing organizations.
Yes.
Yes
Desires one, but exhibits little other than endorsement list.
Reasonably tight coupling
Yes
Yes, the principal investigators are all from central computing. Staff from the Arctic Regions Supercomputing Center will also participate in the project.
Not mentioned in the proposal.
Nothing stands out in this response.
Partner with CCSO (Computing and Communications Services Organization) – an academic computing and networking infrasturucture provider
Yes
Yes – CCSO appears to be just such an organization.
The central campus network organization is represented on this project by one of the principal investigators. Plans for close coordination with the campus infrastructure is evident.
A bit weaker. Citation of Ed Krol of CCSO but little further commitment
Yes
There appears to be a close relationship between CITI and the rest of the central IT infrastructure organizations at the university. Staff are loaned between the organizations for various projects.
This response does not explicitly cite that relationship, but their AFS work suggests that one exists.
Good historic integration between CITI and campus IT, but campus IT not well represented in current proposal.
Not clear.
No
There was no clearly stated relationship between the researchers and the campus computing infrastructure staff.
None demonstrated; have open source experience.
Not mentioned in the proposal.
Not especially.
Condor will enable deployment of PKI enabled middleware to Alliance’s many sites.
Commitment: plan to establish a cross-institutional PKI Laboratory
yes
Yes.
Yes. Principal investigators are from central computing and the computer science department. The proposal shows a good balance of activity on both Computer Science and central computing.
The browser work is very significant, but only if they address the fact that users make decisions based on the logos they see, not based on some certificate they don’t look at or some URL they also tend to ignore. The discussion suggests that they plan to do bi-lateral authentication and authorization of code, but that isn’t spelled out. Their reference to directory services is shared by a number of other respondents, none of whom address the inherent limitations in large directories. That effort is treated as an implementation scaling issue, rather than a usefulness scaling issue.
Seems to understand the focus and needs; good resources
Speed of asymmetric encryption is one of the hindering factor for PKI deployment. Though coprocessor is not a part of PKI, it may significantly solve the performance problems.
One of only a few with interest in Human interface work.
Had some expertise in Authorization Infrastructure.
Has a Professor from IBM's Security Systems.
The CS graduate program in Dartmouth is not established yet (the dept. does not even report the number of graduated PhDs. The undergraduate students however are of outstanding quality.
Yasinsac’s slides cited the 10-risks paper Bruce and I wrote, but the rest of the slides show no use of that content, falling back on PKI Vendor logic instead. The paper was not intended to be an academic paper, but did raise some issues that should have tempered the wording on some of these slides.
Uninteresting
Have specific agenda and name recognition of Dorothy Denning. Also, appears that they will look at practical application for the University Medical center.
The strong references to directories and the decision to centrally store users’ private keys are both suspect in my eyes. I happen to believe that directories lose their usefulness when they get big enough for a human not to be able to find a desired entry. If their work could address that conjecture and either prove or disprove it, it might be interesting to fund. However, if they are so committed to directories as the right way of doing business that they would never approach such a result, then it would seem to me to be a waste of time.
Good people; unfocused proposal
It might be hard for SURA to break free from the NIST focus they currently have, in order to investigate mechanisms antagonistic to NIST’s chosen PKI path or to pursue investigations that could show the Federal PKI effort to be ill-directed.
A concern about this project is the potential difficulty of coordinating a research activity that includes so many different organizations. The focus of the PKI Labs effort itself will be at GTRI so this may be less of an issue than I think but I did not see a clear research agenda proposal from the GTRI PKI Lab in the proposal.
SURA has other works in: Backbone Network System vBNS, Enterprise Directory Security (EDS), Gigabit Multimedia Delivery, QoS, RSVP.
Little prior experience
SURA can leverage a number of researchers across its community of Universities.
Their note about being sensitive to the privacy demands of their users doesn’t say anything about what they plan to do in that area – or what research that might inspire.
Very weak (too bad)
This proposal offers an excellent opportunity to tie into the PKI and security work being done with in the Grid Forum.
It is not clear what U of I would actually plan to study. The two interesting developments cited in (2) above are from existing work. There is potential here, but no plan.
Security lab is one of 14 NSA approved center of excellence in information assurance. The University has very high reputation.
Tie-in with globus/grid is a plus; very different than dartmouth
The Internet2 PKI lab will be housed in CS department, link with under-graduate and graduate students. Three partners to work on PKI.
My overall impression is that these folks are the most likely to implement something – at least based on their previous work. It is not clear that they would do much serious theoretical work on PKI and especially authorization.
Nationally recognized as one of the most innovative in linking undergraduate education and research
CITI worked closely with IBM to identify and correct a security flaw in distributed file system. Works on PKI related to videoconferencing data, QoS and resource reservations
The only reason that I didn?t rate this proposal higher was the smaller set of direct matches between the I2 PKI Labs Draft Research Agenda and the work they describe in the proposal. There is no doubt that they could easily perform the work listed in the I2 PKI Research Agenda.
Did not mention exactly what they intended to do. Have lots of similar projects that could be leveraged.
Excellent shop but they have their own agenda
The KeyNote work is not to be discounted lightly. It is good work. However, this response does not show the level of resources and commitment shown by most other responses.
Trust management may provide a good approach to security on Internet
Grid/NCSA involvement broadens range of applications needing pki
The only reason I didn?t give this proposal a much higher overall score was the lack of described interaction with campus central IT (and central PKI activities). My other concern was that their work might be focused somewhat beyond the one to two year horizon and what I see as a potential issue of how many I2 schools might be willing/able to deploy something like Keynote.
Unfocused
Their proposed budget shows significant funding requests for the PI and co-PIs. The Request for Proposals specifies funding for graduate students and some operational costs only.
Know about Keynote, and want to extend it with Prolog. Description of PKI-based access control and authorization policy language are interesting.
The only really negative note for me was their reference to the performance of basic crypto algorithms. I believe that’s a fruitless research area. Rather, one might look at what class of algorithm to choose – e.g., symmetric vs. asymmetric – or which particular algorithm in a given class to choose.
An investigation of mapping XML <-> ASN.1 seems especially gross, but I would like to hear more about their plans.
Good but unclear they can address the scope of the research agenda